Learning device, learning method, identifying device, identifying method, program, and information processing system

ABSTRACT

A learning device includes: a feature point extracting unit for extracting a feature point from each of multiple generated images made up of a positive image including an identified object, and a negative image excluding the identified object; a feature point feature amount extracting unit for extracting feature point feature amount representing the feature of the feature point from the generated image; a whole feature amount calculating unit for calculating the whole feature amount representing the feature of the whole generated image based on the feature point feature amount of a feature point existing on a feature point selection range determined based on the multiple generated images, of the generated image range; and an identifier generating unit for generating an identifier based on the whole feature amount of the generated image, and a correct answer label representing whether the generated image is the positive image or the negative image.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a learning device, a learning method,an identifying device, an identifying method, a program, and aninformation processing system, and specifically relates to a learningdevice, a learning method, an identifying device, an identifying method,a program, and an information processing system, which are suitablyemployed in the event of identifying whether or not a subject existingon an image is a predetermined object to be identified.

2. Description of the Related Art

There has been an identifying method for performing matching employing atemplate in which an object to be identified is described in a largesense, as an identifying method according to the related art foridentifying (recognizing) from an image obtained by a camera an objectserving as an object to be identified existing on the image thereof.

With this identifying method, a template in which an object to beidentified is described in a large sense, and specifically, a templateof the texture of the whole of an object to be identified is preparedbeforehand, and matching is performed between the template thereof andan image to be identified (image to be processed).

However, with matching employing a template in which an object to beidentified is described, it is difficult to handle partial hiding ordistortion of the object to be identified appearing in an image to beprocessed.

Therefore, an identifying method has been proposed wherein attention ispaid to a local region of an image to be processed, feature amount isextracted from each local region is extracted, and combination of thefeature amount of each local region (group of the feature amount of alocal region), i.e., for example, a vector with the feature amount ofeach local region as an element, is employed to perform identification.

According to the identifying method employing a group of the featureamount of a local region, a problem such as partial hiding or distortionof an object to be identified, which has been hard to handle by theidentifying method employing a template in which an object to beidentified is described in a large sense, is eliminated to some extent,and accordingly, high-precision identification can be performed.

The feature amount of a local region is also used for identification ofthe category of an object in addition to identification of an individualobject. For example, an identifying method for identifying a particularcategory such as a human face using the feature amount of a local regionhas been proposed (e.g., see P. Viola, M. Jones, Robust Real-time FaceDetection, cvpr2001).

Also, with identification of a category, various identifying methodshave been proposed. Examples of identifying methods proposed foridentifying of a category include an identifying method employing a BoF(Bag of Features) histogram (e.g., see G. Csurka, C. Bray, C. Dance, andL. Fan. Visual categorization with bags of keypoint, ECCV2004), and anidentifying method employing correlation of feature amount (e.g., seeJapanese Unexamined Patent Application Publication No. 2007-128195).

For example, with the identifying method employing a BoF histogram,representative feature amount called as Visual codebook is employed,thereby suppressing the dimensions of image expression.

However, in the event of employing such Visual codebook, the informationof the appearance position of the feature amount in an image region islost, and accordingly, deterioration in identifying precision mayresult.

Therefore, in order to deal with such a problem, there has been proposeda method for providing weak position constraint by dividing an imageregion in a grid (lattice) shape (e.g., see S. Lazebnik, C. Schmid, J.Ponce “Beyond Bags of Features: Spatial Pyramid Matching for RecognizingNatural Scene Categories”, CVPR2006).

SUMMARY OF THE INVENTION

However, with the above method for providing weak position restraint bydividing an image region in a grid shape, position restraint as to animage region is common to any object to be identified, and accordingly,precision for identifying an object to be identified deteriorates.

It has been found to be desirable to detect an object to be identifiedwith high precision so as to provide unique position restraint for eachdifferent object to be identified.

A learning device according to a first embodiment of the presentinvention is a learning device including: a feature point extractingunit configured to extract a feature point from each of a plurality ofimages for generation to be used for generating an identifier foridentifying whether or not a subject existing on an image is apredetermined object to be identified, which are made up of a positiveimage where the object to be identified exists, and a negative imagewhere the object to be identified does not exist; a feature pointfeature amount extracting unit configured to extract feature pointfeature amount representing the feature of the feature point from theimage for generation; a whole feature amount calculating unit configuredto calculate whole feature amount representing the feature of the wholeof the image for generation based on the feature point feature amount ofa feature point existing on a feature point selection range determinedbased on the plurality of images for generation, of the whole range onthe image for generation; and an identifier generating unit configuredto generate the identifier based on the whole feature amount of theimage for generation, and a correct answer label representing whetherthe image for generation is the positive image or the negative image.

With the learning device according to the first embodiment, there mayfurther be provided a range determining unit configured to determine thefeature point selection range for selecting the feature point featureamount to be used for calculation for calculating each of a plurality ofdimensional feature amounts that is each dimensional element of thewhole feature amount to be represented by a plurality of dimensionalvectors based on the plurality of images for generation; and a storageunit configured to store feature amount for calculation to be used forthe calculation as to the feature point feature amount, and the featurepoint selection range determined by the range determining unit in acorrelated manner; with the whole feature amount calculating unitcalculating each of a plurality of dimensional feature amounts making upthe whole feature amount by the calculation between the feature amountfor calculation, and the feature point feature amount of a feature pointexisting on the feature point selection range of the image forgeneration correlated with the feature amount for calculation.

With the learning device according to the first embodiment, the rangedetermining unit may set a plurality of candidate ranges that arecandidates of the feature point selection range, and based on an errormap including error information representing a degree of mistakingidentification regarding whether the plurality of images for generationare the positive images or the negative images, and may determine acandidate range where the error information becomes the minimum as thefeature point selection range for each of the candidate ranges.

With the learning device according to the first embodiment, the rangedetermining unit may calculate the error information for each of thecandidate ranges based on the dimensional feature amount to becalculated by the calculation between the feature point feature amountof a feature point existing on the candidate range and the featureamount for calculation, and may determine the feature point selectionrange based on the error map including the calculated error information.

With the learning device according to the first embodiment, the wholefeature amount calculating unit may calculate a correlation valuerepresenting correlation between the feature amount for calculation, andthe feature point feature amount of a feature point existing on thesearch range as dimensional feature amount making up the whole featureamount.

With the learning device according to the first embodiment, theidentifier generating unit may generate, of a plurality of dimensionalfeature amounts that is each dimensional element of the whole featureamount to be represented with a plurality of dimensional vectors, theidentifier for performing identification using the dimensional featureamount that reduces error information representing a degree of mistakingidentification of the positive image and the negative image, anddimensional information representing the dimension of the dimensionalfeature amount that reduces the error information.

A learning method according to the first embodiment of the presentinvention is a learning method of a learning device for learning anidentifier for identifying a predetermined object to be identified, withthe learning device including a feature point extracting unit, a featurepoint feature amount extracting unit, a whole feature amount calculatingunit, and an identifier generating unit, the learning method includingthe steps of: extracting, with the feature point extracting unit, afeature point from each of a plurality of images for generation to beused for generating an identifier for identifying whether or not asubject existing on an image is a predetermined object to be identified,which are made up of a positive image where the object to be identifiedexists, and a negative image where the object to be identified does notexist; extracting, with the feature point feature amount extractingunit, feature point feature amount representing the feature of thefeature point from the image for generation; calculating, with the wholefeature amount calculating unit, whole feature amount representing thefeature of the whole of the image for generation based on the featurepoint feature amount of a feature point existing on a feature pointselection range determined based on the plurality of images forgeneration, of the whole range on the image for generation; andgenerating, with the identifier generating unit, the identifier based onthe whole feature amount of the image for generation, and a correctanswer label representing whether the image for generation is thepositive image or the negative image.

A first program according to the first embodiment of the presentinvention is a program causing a computer to serve as: a feature pointextracting unit configured to extract a feature point from each of aplurality of images for generation to be used for generating anidentifier for identifying whether or not a subject existing on an imageis a predetermined object to be identified, which are made up of apositive image where the object to be identified exists, and a negativeimage where the object to be identified does not exist; a feature pointfeature amount extracting unit configured to extract feature pointfeature amount representing the feature of the feature point from theimage for generation; a whole feature amount calculating unit configuredto calculate whole feature amount representing the feature of the wholeof the image for generation based on the feature point feature amount ofa feature point existing on a feature point selection range determinedbased on the plurality of images for generation, of the whole range onthe image for generation; and an identifier generating unit configuredto generate the identifier based on the whole feature amount of theimage for generation, and a correct answer label representing whetherthe image for generation is the positive image or the negative image.

According to the first embodiment of the present invention, a featurepoint is extracted from each of a plurality of images for generation tobe used for generating an identifier for identifying whether or not asubject existing on an image is a predetermined object to be identified,which are made up of a positive image where the object to be identifiedexists, and a negative image where the object to be identified does notexist, feature point feature amount representing the feature of thefeature point is extracted from the image for generation; whole featureamount representing the feature of the whole of the image for generationis calculated based on the feature point feature amount of a featurepoint existing on a feature pointy selection range determined based onthe plurality of images for generation, of the whole range on the imagefor generation, and the identifier is generated based on the wholefeature amount of the image for generation, and a correct answer labelrepresenting whether the image for generation is the positive image orthe negative image.

An identifying device according to a second embodiment of the presentinvention is an identifying device including: a feature point extractingunit configured to extract a feature point from an image to be processedserving as a processing object for identifying whether or not a subjectexisting on an image is a predetermined object to be identified; afeature point feature amount extracting unit configured to extractfeature point feature amount representing the feature of the featurepoint from the image to be processed; a whole feature amount calculatingunit configured to calculate the whole feature amount representing thefeature of the whole of the image to be processed based on the featurepoint feature amount of a feature point existing on a feature pointselection range determined based on a plurality of images for generationto be used for learning for generating an identifier for identifyingeach different object to be identified of the whole range on the imageto be processed; and an identifying unit configured to identify, basedon an identifier for identifying whether or not a subject existing on animage is a predetermined object to be identified, and the whole featureamount, whether or not a subject existing on the image to be processedis a predetermined object to be identified.

With the identifying device according to the second embodiment, theremay further be provided a storage unit configured to store featureamount for calculation to be used for calculation for calculating eachof a plurality of dimensional feature amounts that is each dimensionalelement of the whole feature amount represented by a plurality ofdimensional vectors, and the feature point selection range for selectingthe feature point feature amount to be used for the calculation with thefeature amount for calculation in a correlated manner; with the wholefeature amount calculating unit calculating each of a plurality ofdimensional feature amounts making up the whole feature amount by thecalculation between the feature amount for calculation, and the featurepoint feature amount of a feature point existing on the feature pointselection range of the image to be processed correlated with the featureamount for calculation.

With the identifying device according to the second embodiment, thewhole feature amount calculating unit may perform the calculation forcalculating a correlation value representing correlation between thefeature amount for calculation, and the feature point feature amount ofa feature point existing on the search range as dimensional featureamount making up the whole feature amount.

With the identifying device according to the second embodiment, thewhole feature amount calculating unit may calculate whole feature amountrepresenting the whole of the image to be processed, which is made up ofa plurality of dimensional information, based on the feature pointfeature amount of a feature point existing on the feature pointselection range; with the identifying unit identifying whether or not asubject existing on the image to be processed is a predetermined objectto be identified by providing predetermined dimensional feature amountof the plurality of dimensional feature amounts making up the wholefeature amount to the identifying device as input.

With the identifying device according to the second embodiment, theidentifying unit may provide the dimensional feature amount of thedimension represented by dimensional information, of the plurality ofdimensional feature amounts making up the whole feature amount, to anidentifier for identifying whether or not a subject existing on an imageis a predetermined object to be identified, as input, therebyidentifying whether or not a subject appears on the image to beprocessed is a predetermined object to be identified; with theidentifier performing identification using, of the plurality ofdimensional information representing the whole feature amount, thedimensional feature amount that reduces error information representing adegree of mistaking identification regarding whether the object to beidentified is a positive image existing on an image, or a negative imagenot existing on the image; and with the dimensional informationrepresenting the dimension of the dimensional feature amount thatreduces the error information.

An identifying method according to the second embodiment of the presentinvention is an identifying method of an identifying device foridentifying whether or not a subject existing on an image is apredetermined object to be identified, with the identifying deviceincluding a feature point extracting unit, a feature point featureamount extracting unit, a whole feature amount calculating unit, and anidentifying unit, the identifying method including the steps of:extracting, with the feature point extracting unit, a feature point froman image to be processed serving as a processing object for identifyingwhether or not a subject existing on an image is a predetermined objectto be identified; extracting, with the feature point feature amountextracting unit, feature point feature amount representing the featureof the feature point from the image to be processed; calculating, withthe whole feature amount calculating unit, whole feature amountrepresenting the feature of the whole of the image to be processed basedon the feature point feature amount of a feature point existing on afeature point selection range determined based on the plurality ofimages for generation to be used for learning for generating anidentifier which identifies each different object to be identified, ofthe whole range on the image to be processed; and identifying, with theidentifier generating unit, whether or not a subject existing on theimage to be processed is a predetermined object to be identified, basedon an identifier for identifying whether or not a subject exiting on animage is a predetermined object to be identified, and the whole featureamount.

A second program according to the second embodiment of the presentinvention is a program causing a computer to serve as: a feature pointextracting unit configured to extract a feature point from an image tobe processed serving as a processing object for identifying whether ornot a subject existing on an image is a predetermined object to beidentified; a feature point feature amount extracting unit configured toextract feature point feature amount representing the feature of thefeature point from the image to be processed; a whole feature amountcalculating unit configured to calculate whole feature amountrepresenting the feature of the whole of the image to be processed basedon the feature point feature amount of a feature point existing on afeature point selection range determined based on the plurality ofimages for generation to be used for learning for generating anidentifier which identifies each different object to be identified, ofthe whole range on the image to be processed; and an identifying unitconfigured to identify whether or not a subject existing on the image tobe processed is a predetermined object to be identified, based on anidentifier for identifying whether or not a subject exiting on an imageis a predetermined object to be identified, and the whole featureamount.

According to the second embodiment of the present invention, a featurepoint is extracted from an image to be processed serving as a processingobject for identifying whether or not a subject existing on an image isa predetermined object to be identified, feature point feature amountrepresenting the feature of the feature point is extracted from theimage to be processed, whole feature amount representing the feature ofthe whole of the image to be processed is calculated based on thefeature point feature amount of a feature point existing on a featurepoint selection range determined based on the plurality of images forgeneration to be used for learning for generating an identifier whichidentifies each different object to be identified, of the whole range onthe image to be processed, and regarding whether or not a subjectexisting on the image to be processed is a predetermined object to beidentified is identified based on an identifier for identifying whetheror not a subject exiting on an image is a predetermined object to beidentified, and the whole feature amount.

An information processing system according to a third embodiment of thepresent invention is an information processing system including: alearning device configured to learn an identifier for identifying apredetermined object to be identified; and an identifier configured toidentify whether or not a subject existing on an image is apredetermined object to be identified, using the identifier; with thelearning device including a first feature point extracting unitconfigured to extract a feature point from each of a plurality of imagesfor generation to be used for generating an identifier for identifyingwhether or not a subject existing on an image is a predetermined objectto be identified, which are made up of a positive image where the objectto be identified exists, and a negative image where the object to beidentified does not exist, a first feature point feature amountextracting unit configured to extract feature point feature amountrepresenting the feature of the feature point from the image forgeneration, a first whole feature amount calculating unit configured tocalculate whole feature amount representing the feature of the whole ofthe image for generation based on the feature point feature amount of afeature point existing on a feature point selection range determinedbased on the plurality of images for generation, of the whole range onthe image for generation, and an identifier generating unit configuredto generate the identifier based on the whole feature amount of theimage for generation, and a correct answer label representing whetherthe image for generation is the positive image or the negative image;and with the identifier including a second feature point extracting unitconfigured to extract a feature point from an image to be processedserving as a processing object for identifying whether or not a subjectexisting on an image is a predetermined object to be identified, asecond feature point feature amount extracting unit configured toextract feature point feature amount representing the feature of thefeature point from the image to be processed, a second whole featureamount calculating unit configured to calculate the whole feature amountrepresenting the feature of the whole of the image to be processed basedon the feature point feature amount of a feature point existing on afeature point selection range determined based on a plurality of imagesfor generation to be used for learning by the learning device, of thewhole range on the image to be processed, and an identifying unitconfigured to identify, based on the identifier generated by theidentifier generating unit, and the whole feature amount of the image tobe processed, whether or not a subject existing on the image to beprocessed is a predetermined object to be identified.

With the third embodiment of the present invention, according to thelearning device, a feature point is extracted from each of a pluralityof images for generation to be used for generating an identifier foridentifying whether or not a subject existing on an image is apredetermined object to be identified, which are made up of a positiveimage where the object to be identified exists, and a negative imagewhere the object to be identified does not exist, feature point featureamount representing the feature of the feature point is extracted fromthe image for generation, whole feature amount representing the featureof the whole of the image for generation is calculated based on thefeature point feature amount of a feature point existing on a featurepoint selection range determined based on the plurality of images forgeneration, of the whole range on the image for generation, and theidentifier is generated based on the whole feature amount of the imagefor generation, and a correct answer label representing whether theimage for generation is the positive image or the negative image.Subsequently, according to the identifying device, a feature point isextracted from an image to be processed serving as a processing objectfor identifying whether or not a subject existing on an image is apredetermined object to be identified, feature point feature amountrepresenting the feature of the feature point is extracted from theimage to be processed, the whole feature amount representing the featureof the whole of the image to be processed is calculated based on thefeature point feature amount of a feature point existing on a featurepoint selection range determined based on a plurality of images forgeneration to be used for learning by the learning device, of the wholerange on the image to be processed, and based on the identifiergenerated by the identifier generating unit, and the whole featureamount of the image to be processed, regarding whether or not a subjectexisting on the image to be processed is a predetermined object to beidentified is identified.

According to the present invention, unique position restraint isprovided for each object to be identified, whereby an object to beidentified can be detected in a more accurate manner.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a first configuration example ofa learning device to which an embodiment of the present invention hasbeen applied;

FIG. 2 is a block diagram illustrating a configuration example of ashared code book generating unit;

FIG. 3 is a diagram for describing an example of processing to beperformed by a feature amount replacing unit and a quantizing unit;

FIG. 4 is a diagram illustrating an example of a shared code book;

FIGS. 5A through 5C are diagrams for describing an example of processingto be performed by the quantizing unit;

FIG. 6 is a diagram for describing an example of processing to beperformed by the whole feature amount calculating unit in FIG. 1;

FIG. 7 is a diagram for describing an example of an identifiergenerating method;

FIG. 8 is a flowchart for describing learning processing to be performedby the learning device in FIG. 1;

FIG. 9 is a flowchart for describing shard code book generationprocessing;

FIG. 10 is a flowchart for describing code book generation processing;

FIG. 11 is a flowchart for describing identifier generation processing;

FIG. 12 is a block diagram illustrating a configuration example of afirst identifying device for identifying an object to be identifiedusing an identifier generated by learning of the learning device in FIG.1;

FIG. 13 is a flowchart for describing identification processing to beperformed by the identifying device in FIG. 12;

FIG. 14 is a block diagram illustrating a second configuration exampleof a learning device to which an embodiment of the present invention hasbeen applied;

FIG. 15 is a diagram for describing an example of processing to beperformed by a range determining unit;

FIG. 16 is a diagram for describing an example of processing to beperformed by the whole feature amount calculating unit in FIG. 14;

FIG. 17 is a flowchart for describing learning processing to beperformed by the learning device in FIG. 14;

FIG. 18 is a flowchart for describing range determination processing;

FIG. 19 is a block diagram illustrating a configuration example of asecond identifying device for identifying an object to be identifiedusing an identifier generated by learning of the learning device in FIG.14;

FIG. 20 is a flowchart for describing identification processing to beperformed by the identifier in FIG. 19; and

FIG. 21 is a block diagram illustrating a configuration example of acomputer.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereafter, modes (hereafter, referred to as embodiments) for carryingout the present invention will be described. Note that description willbe made in the following sequence.

1. First Embodiment (example in which feature point feature amount iscommonly used by a different object to be identified)

2. Second Embodiment (example in which a different range is used foreach object to be identified)

3. Modifications

First Embodiment Configuration Example of Learning Device 1

FIG. 1 illustrates a configuration example of a learning device 1 towhich a first embodiment of the present invention has been applied.

This learning device 1 uses an image for learning to generate anidentifier (function) for identifying whether or not a subject existingon the image is a predetermined object to be identified, andlater-described dimensional information.

Here, the image for learning is an image to be used for generation(learning) of an identifier, and includes multiple model images,multiple primitive images, and multiple images for generation.

The model images are images on which a subject that can be served as anobject to be identified (e.g., note-type personal computer (notebookcomputer), automobile, etc.) exists. Also, the primitive images areimages including a plurality of feature amounts not depending on aparticular object to be identified, i.e., images to be used forextracting various types of feature amount regardless of the featureamount of a particular object to be identified.

As for an example of the primitive images, an image on which anartificial object or natural object such as scenery exists is employed.

Also, the images for generation include both of a positive image onwhich an object (e.g., notebook computer) to be identified exists, and anegative image on which an object to be identified does not exist.

Further, the images for generation have added thereto a correct answerlabel. This correct answer label exists for each image for generation,and represents whether each image for generation is either a positiveimage or a negative.

Specifically, this learning device 1 generates a shared code bookwherein the feature amount extracted from multiple primitive images isshared as the feature amount of a subject (e.g., notebook computer,automobile, etc.) that can be served as an object to be identified.

Subsequently, the learning device 1 uses the generated shared code bookto generate an identifier for identifying whether or not a subjectexisting on an image is a predetermined object to be identified (e.g.,automobile), and the corresponding dimensional information.

The learning device 1 is configured of an input unit 21, a shared codebook generating unit 22, a shared code book storage unit 23, a featurepoint extracting unit 24, a feature amount extracting unit 25, a wholefeature amount calculating unit 26, and an identifier generating unit27.

Multiple model images, multiple primitive images, and multiple imagesfor generation are supplied to the input unit 21 as images for learning.The input unit 21 inputs (supplies) the supplied multiple model imagesand multiple primitive images to the shared code book generating unit22.

Also, the input unit 21 inputs the supplied multiple images forgeneration to the feature point extracting unit 24.

The shared code book generating unit 22 generates a shared code bookwhere the feature amount of a subject that can be served as an object tobe identified is shared, based on the multiple model images and multipleprimitive images from the input unit 21, and supplies and stores this tothe shared code book storage unit 23.

Note that the details of the shared code book generating unit 22 will bedescribed later with reference to FIG. 2.

The shared code book storage unit 23 stores the shared code book fromthe shared code book generating unit 22.

The feature point extracting unit 24 sequentially selects each of themultiple images for generation from the input unit 21 as an image forgeneration of interest, and extracts a feature point from the image forgeneration of interest. Subsequently, the feature point extracting unit24 supplies the extracted feature point to the feature amount extractingunit 25 along with the image for generation of interest.

Here, the local information of an image is frequently included in acorner point, so the feature point extracting unit 24 extracts (a pixelserving as) a corner point as a feature point.

Extraction of a corner point can be performed using a Harris cornerdetector. With the Harris corner detector, if we say that the pixelvalue (e.g., luminance) of the pixel of a certain position (x, y) isrepresented with I(x, y), a pixel of which the two eigen values of thesecondary moment L of a luminance gradient obtained by the followingExpression (1) are equal to or greater than a predetermined threshold isdetected as a corner point.

$\begin{matrix}{L = \begin{bmatrix}\left( \frac{\partial I}{\partial x} \right)^{2} & {\left( \frac{\partial I}{\partial x} \right)\left( \frac{\partial I}{\partial y} \right)} \\{\left( \frac{\partial I}{\partial x} \right)\left( \frac{\partial I}{\partial y} \right)} & \left( \frac{\partial I}{\partial y} \right)^{2}\end{bmatrix}} & (1)\end{matrix}$

Note that, in Expression (1), the pixel value I(x, y) is described as Iby omitting (x, y).

Also, with the feature point extracting unit 24, in addition to a cornerpoint, for example, a pixel serving as an edge, or a pixel in apredetermined fixed position may be employed as a feature point.

The feature amount extracting unit 25 extracts feature point featureamount representing the feature of a feature point from the featurepoint extracting unit 24 from the image for generation of interest fromthe feature point extracting unit 24, and supplies this to the wholefeature amount calculating unit 26. Also, the feature amount extractingunit 25 extracts the correct answer label added to the image forgeneration of interest from the feature point extracting unit 24, andsupplies this to the whole feature amount calculating unit 26.

The whole feature amount calculating unit 26 calculates whole featureamount representing the feature of the whole of the image for generationof interest thereof from the feature point feature amount from thefeature amount extracting unit 25 based on the shared code book storedin the shared code book storage unit 23.

Here, the whole feature amount is represented with, for example,multiple dimensional vectors (vectors including multiple values as anelement).

The whole feature amount calculating unit 26 supplies the whole featureamount of the calculated image for generation of interest to theidentifier generating unit 27 along with the correct answer label of theimage for generation of interest from the feature amount extracting unit25.

In response to the corresponding whole feature amount and correct answerlabel regarding each of the multiple images for generation from thewhole feature amount calculating unit 26, the identifier generating unit27 uses the whole feature amount of each of the multiple images forgeneration, and the correct answer label of each of the multiple imagesfor generation to generate an identifier (perform learning to calculatea parameter to stipulate an identifier).

Here, if we say that the elements of the multiple dimensional vectorsserving as the whole feature amount are taken as dimensional featureamount, the whole feature amount is made up of a plurality (a numberequal to the dimension of vector) of dimensional feature amount.

The identifier generating unit 27 generates an identifier which uses notthe entire dimensional feature amount making up the whole feature amountbut part of the dimensional feature amount selected out of thedimensional feature amount making up the whole feature amount to performidentification.

Also, the identifier generating unit 27 generates informationrepresenting the dimension of the dimensional feature amount to be usedfor identification by this identifier (information representing whatnumber element of vector serving as the whole feature amount thedimensional feature amount is) as dimensional information.

Configuration Example of Shared Code Book Generating Unit 22

FIG. 2 illustrates a detailed configuration example of the shared codebook generating unit 22.

This shared code book generating unit 22 is configured of a featurepoint extracting unit 41, a feature amount extracting unit 42, a codebook generating unit 43, a code book storage unit 44, a feature pointextracting unit 45, a feature point extracting unit 46, a feature amountreplacing unit 47, and a quantizing unit 48.

The multiple primitive images are input from the input unit 21 to thefeature point extracting unit 41. The feature point extracting unit 41sequentially takes each of the multiple primitive images from the inputunit 21 as a primitive image of interest. Subsequently, the featurepoint extracting unit 41 extracts a feature point from the primitiveimage of interest, and supplies this to the feature amount extractingunit 42 along with the primitive image of interest.

The feature amount extracting unit 42 similarly extracts feature pointfeature amount representing the feature of the feature point from thefeature point extracting unit 41 from the primitive image of interestfrom the feature point extracting unit 41 to supply this to the codebook generating unit 43.

Specifically, the feature amount extracting unit 42 supplies theplurality of feature point amounts extracted from each of the multipleprimitive images to be selected as a primitive image of interest at thefeature point extracting unit 41 to the code generating unit 43.

The code book generating unit 43 subjects the plurality of feature pointfeature amounts to be supplied from the feature amount extracting unit42 to grouping into several groups, for example, by a clustering methodsuch as the k-means method in the feature amount space. Subsequently,the code book generating unit 43 determines, for every several groups tobe obtained by grouping, the centroid of feature point feature amountbelonging to the groups thereof as feature point feature amountrepresenting the groups.

The code book generating unit 43 correlates the feature point featureamount determined for every several groups with a discriminator Cbn(n=1,2, . . . , N) for uniquely identifying the feature point feature amountthereof. Also, the code book generating unit 43 generates a code bookincluding the feature point feature amount correlated with thediscriminator Cbn to supply and store this to the code book storage unit44.

The code book storage unit 44 stores the code book in which thediscriminator Cbn and the feature point feature amount to be identifiedby the discriminator Cbn are correlated.

The multiple model images are input to the feature point extracting unit45 from the input unit 21. The feature point extracting unit 45 subjectsthe multiple model images to be input from the input unit 21 to groupinginto multiple groups by the category of a subject existing on the modelimages. Note that the model images include category informationrepresenting the category of a subject existing on the images, and thefeature point extracting unit 45 is arranged to perform grouping basedon the category information included in the model images.

Subsequently, the feature point extracting unit 45 sequentially selectseach of the multiple groups grouped by category as a group of interest.

Further, the feature point extracting unit 45 sequentially selects eachof the multiple images included in the group of interest as a modelimage of interest. Subsequently, the feature point extracting unit 45extracts a feature point from the model image of interest to supply thisto the feature amount extracting unit 46 along with the model image ofinterest.

The feature amount extracting unit 46 similarly extracts feature pointfeature amount representing the feature of the feature point from thefeature point extracting unit 45 from the model image of interest fromthe feature point extracting unit 45. Subsequently, the feature amountextracting unit 46 correlates the feature point from the feature pointextracting unit 45 with the extracted feature point feature amount.

Thus, the feature amount extracting unit 46 obtains an initial featureamount pool including the feature point feature amount correlated withthe corresponding feature point, included in the plurality of featurepoint feature amounts extracted from each of the multiple model imagesincluded in the group of interest.

The feature amount extracting unit 46 supplies the obtained initialfeature amount pool to the feature amount replacing unit 47.

The feature amount replacing unit 47 replaces the feature point featureamount included in the initial feature amount pool from the featureamount extracting unit 46 with the discriminator Cbn included in thecode book stored in the code book storage unit 44.

Specifically, for example, the feature amount replacing unit 47 readsout the code book from the code book storage unit 44. Subsequently, thefeature amount replacing unit 47 replaces the feature point featureamount including the initial feature amount pool from the feature amountextracting unit 46 with the discriminator Cbn for identifying the mostsimilar feature point feature amount of the plurality of feature pointfeature amounts including the read code book.

In this way, the feature amount replacing unit 47 replaces each of theplurality of feature point feature amounts included in the initialfeature amount pool from the feature amount extracting unit 46 with thediscriminator Cbn included in the code book stored in the code bookstorage unit 44.

Thus, the feature amount replacing unit 47 generates, from the initialfeature amount pool including the feature point feature amountcorrelated with a feature point, a feature amount pool including thediscriminator Cbn correlated with the feature point. The feature amountreplacing unit 47 supplies the generated feature amount pool to thequantizing unit 48.

Also, the feature amount replacing unit 47 supplies the code book readout from the code book storage unit 44 to the quantizing unit 48.

The quantizing unit 48 quantizes the feature point correlated with thediscriminator Cbn included in the feature pool from the feature amountreplacing unit 47 to obtain a quantized feature amount pool obtained asa result thereof.

Also, the quantizing unit 48 obtains shared information to be obtainedby grouping the same discriminators Cbn of the discriminators Cbnincluded in the obtained quantized feature amount pool.

Subsequently, the quantizing unit 48 supplies and stores a combinationof the shared information obtained by category and the code book fromthe feature amount replacing unit 47 to the shared code book storageunit 23 as a shared code book.

Example of Processing to be Performed by Feature Amount Replacing Unit47 and Quantizing Unit 48

Next, FIG. 3 illustrates an example of processing to be performed by thefeature amount replacing unit 47 and quantizing unit 48.

Note that the code book 61 illustrated in FIG. 3 is a code book storedin the code book storage unit 44, and includes the discriminator Cbn andfeature point feature amount fvec to be identified by the discriminatorCbn.

The feature amount replacing unit 47 reads out the code book 61 from thecode book storage unit 44. Also, the feature amount replacing unit 47extracts predetermined feature point feature amount from an initialfeature amount pool 62 including the plurality of feature point featureamounts in a predetermined category (illustrated with a cross) to besupplied from the feature amount extracting unit 46 to sequentiallyselect this as feature point feature amount of interest fvec′.

Subsequently, the feature amount replacing unit 47 replaces the featurepoint feature amount of interest fvec′ with the discriminator Cbnincluded in the read code book 61. Specifically, for example, thefeature amount replacing unit 47 replaces the feature point featureamount of interest fvec′ with the discriminator Cbn for identifying thefeature point feature amount fvec most similar to the feature pointfeature amount of interest fvec′ of the plurality of feature pointfeature amounts included in the read code book 61.

The feature amount replacing unit 47 replaces all of the plurality offeature point feature amounts included in the initial feature amountpool 62 with the discriminator Cbn by selecting all of the plurality offeature point feature amounts included in the initial feature amountpool 62 as the feature point feature amount of interest fvec′.

Thus, the feature amount replacing unit 47 converts the initial featureamount pool 62 in which the feature point feature amount of interestfvec′ is correlated with the corresponding feature point (x, y) into afeature amount pool 63 in which the discriminator Cbn is correlated withthe feature point (x, y) corresponding to the feature point featureamount of interest fvec′.

The feature amount replacing unit 47 supplies the converted featureamount pool 63 to the quantizing unit 48 along with the code book 61read out from the code book storage unit 44.

The quantizing unit 48 performs quantization to convert several featurepoints included in the feature amount pool 63 from the feature amountreplacing unit 47 into one feature point, and obtains a quantizedfeature amount pool 64 obtained as a result thereof. Note that thedetails of the quantization to be performed by the quantizing unit 48will be described later with reference to FIG. 4.

Also, the quantizing unit 48 groups the same discriminators Cbn of thediscriminators Cbn included in the obtained quantized feature amountpool 64, and supplies and stores shared information 65 to be obtained asa result thereof to the shared code book storage unit 23.

Specifically, for example, the quantizing unit 48 groups the featurepoint with which the same discriminators Cbn are correlated of thediscriminators Cbn included in the quantized feature amount pool 64,thereby generating shared information 65 with which a single or multiplefeature points are correlated, for every discriminator Cbn.

Subsequently, the quantizing unit 48 supplies and stores a combinationbetween the generated shared information 65 and the code book 61 fromthe feature amount replacing unit 47 to the shared code book storageunit 23 as a shared code book 81.

Note that the shared information 65 is generated for every multiplecategories based on the initial feature amount pool 62 in the categorythereof. Accordingly, a shared code book 81 configured of sharedinformation 65 _(m) generated for every multiple categories, and thecode book 61 is stored in the shared code book storage unit 23.

Also, as described above, the quantizing unit 48 has been arranged toobtain the shared information 65 by grouping the same discriminators Cbnof the discriminators Cbn included in the quantized feature amount pool64, but is not restricted to this.

Specifically, for example, the quantizing unit 48 can obtain the sharedinformation 65 by grouping the same discriminators Cbn of thediscriminators Cbn included in not the quantized feature amount pool 64but the feature amount pool 63 without quantizing the feature amountpool 63 from the feature amount replacing unit 47.

Example of Shared Code Book

Next, FIG. 4 illustrates an example of the shared code book 81 to bestored in the shared code book 81 generated by the shared code bookgenerating unit 22 and stored in the shared code book storage unit 23.

The shared code book 81 illustrated in FIG. 4 is made up of sharedinformation 65 ₁ in a category A to which a notebook personal computerbelongs, and shared information 65 ₂ in category B to which anautomobile belongs.

The shared information 65 ₁ is shared information generated based on theinitial feature amount pool 62 including the plurality of feature pointfeature amounts in the category A.

With the shared information 65 ₁, a discriminator Cb_(z) is correlatedwith feature points (x₁, y₁) and (x₂, y₂). Also, with the sharedinformation 65 ₁, a discriminator Cb₃₀ is correlated with a featurepoint (x₃, y₃).

The shared information 65 ₂ is shared information generated based on theinitial feature amount pool 62 including the plurality of feature pointfeature amounts in the category B.

With the shared information 65 ₂, a discriminator Cb₃₀ is correlatedwith feature points (x₄, y₄) and (x₅, y₅). Also, with the sharedinformation 65 ₂, a discriminator Cb₄₅ is correlated with feature points(x₆, y₆) and (x₇, y₇).

Specifically, with a combination 61 a between the feature point featureamount fvec and the discriminator Cb₃₀ for identifying the feature pointfeature amount fvec, the discriminator Cb₃₀ of the combination 61 a iscorrelated with the feature point (x₃, y₃) in the category A, and thefeature points (x₆, y₆) and (x₇, y₇) in the category B.

Accordingly, even in either case of the case of identifying category A,and the case of identifying category B, the feature point feature amountfvec to be identified by the discriminator Cb₃₀ is commonly employed.

Also, for example, a feature point extracted from a model image in thecategory A, and a feature point extracted from a model image in thecategory B generally differ. Therefore, the feature point (x₃, y₃)correlated with the discriminator Cb₃₀ in the category A, and thefeature points (x₄, y₄) and (x₅, y₅) correlated with the discriminatorCb₃₀ in the category B differ.

Accordingly, in the event of identifying the category A, and in theevent of identifying the category B, a different feature point isemployed.

Example of Quantization to be Performed by Quantizing Unit 48

Next, the details of the quantization to be performed by the quantizingunit 48 will be described with reference to FIGS. 5A through 5C. FIGS.5A through 5C are diagrams for describing the details of quantization tobe performed by the quantizing unit 48.

FIG. 5A illustrates, for example, four model images on which a notebookpersonal computer exits as a subject. Note that, with the four modelimages illustrated in FIG. 5A, a corner point (illustrated with a cross)is illustrated as a feature point for example.

FIG. 5B illustrates a feature point frequency image 101 configured of apixel with a frequency (degree) of a feature point existing as aluminance value in the four model images illustrated in FIG. 5A.

FIG. 5C illustrates an example of a scene wherein a pixel with a greaterluminance value than those of peripheral pixels has been selected as afeature point (illustrated with a cross) in the feature point frequencyimage 101 illustrated in FIG. 5B.

The quantizing unit 48 sequentially selects a feature point included inthe feature amount pool 63 from the feature amount replacing unit 47,i.e., for example, each of corner points (illustrated with a cross) ofeach of the model images illustrated in FIG. 5A, as a feature point ofinterest.

Also, with the quantizing unit 48, there is prepared the feature pointfrequency image 101 having the same size as the model images illustratedin FIG. 5A, and each pixel of which has a luminance value of zero.

Subsequently, the quantizing unit 48 votes (adds) a value correspondingto Gauss weight (weight following a gauss distribution) to the pixelcorresponding to the feature point of interest, and the luminance valueof a pixel around that pixel, of multiple pixels making up the preparedfeature point frequency image 101.

Thus, with the feature point frequency image 101, 1 is voted to theluminance value of the pixel corresponding to the feature point ofinterest, and a smaller value is voted to a pixel existing in a positionapart from the position of that pixel as the distance therebetweenbecomes longer.

The quantizing unit 48 performs a vote by selecting all of the featurepoints included in the feature amount pool 63 as a feature point ofinterest, thereby generating the feature point frequency image 101 witha feature point having different luminance according to a frequencywhere the feature point exists.

Subsequently, the quantizing unit 48 performs non-maximum suppressionprocessing wherein a feature point corresponding to a pixel having agreater luminance value than that of a peripheral pixel is extracted asa representative feature point as to the generated feature pointfrequency image 101.

Specifically, for example, the quantizing unit 48 extracts, of theluminance values of pixels making up the feature point frequency image101, the position of a pixel having the maximum luminance value as arepresentative feature point. Subsequently, the quantizing unit 48excludes the region around the representative feature point thereofincluding the extracted representative feature point from a processingobject to extract a new representative feature point.

Next, the quantizing unit 48 extracts, of the luminance value of a pixelmaking up the feature point frequency image 101 after exclusion of theregion around the extracted representative feature point from theprocessing object, the position of a pixel having the maximum luminancevalue as a representative feature point. Subsequently, the quantizingunit 48 excludes the region around the representative feature pointthereof including the extracted representative feature point from theprocessing object, and thereafter the same processing is repeated.

In this way, in the event of having extracted a predeterminedrepresentative feature points (illustrated with a cross) from thefeature point frequency image 101 as illustrated in FIG. 5C, thequantizing unit 48 ends the non-maximum suppression processing.

The quantizing unit 48 quantizes the feature points included in thefeature amount pool 63 into one of the multiple representative featurepoints extracted by the non-maximum suppression processing.

Specifically, for example, the quantizing unit 48 sequentially selectsthe feature points included in the feature amount pool 63 as a featurepoint of interest. Subsequently, the quantizing unit 48 performsquantization to convert the feature point of interest into arepresentative feature point existing in a position closest to thefeature point of interest, of the multiple representative featurepoints.

The quantizing unit 48 quantizes all of the feature points included inthe feature amount pool 63 into one of the multiple representativefeature points, and obtains a quantized feature amount pool 64 obtainedby the quantization thereof.

Subsequently, the quantizing unit 48 groups the multiple discriminatorsCbn included in the quantized feature amount pool 64 between the samediscriminators Cbn as described above. Also, the quantizing unit 48supplies and stores a combination between the shared information 65 tobe obtained by grouping thereof, and the code book 61 from the featureamount replacing unit 47 to the shared code book storage unit 23 as theshared code book 81.

Example of Calculation of Correlation Values

Next, description will be made regarding an example of processing forcalculating the whole feature amount to be performed by the wholefeature amount calculating unit 26, with reference to FIG. 6.

In FIG. 6, as the whole feature amount for generating an identifier anddimensional information for identifying whether or not an object to beidentified belongs to the category B of automobiles, the whole featureamount calculating unit 26 for calculating a correlation value will bedescribed, for example.

Specifically, FIG. 6 illustrates an example in the event that the wholefeature amount calculating unit 26 uses the shared information 65 ₂ inthe category B of automobiles, and the code book 61 in the shared codebook 81 stored in the shared code book storage unit 23 to calculate thewhole feature amount of an image for generation.

The upper side in FIG. 6 illustrates, of the plurality of feature pointfeature amounts included in the code book 61 included in the shared codebook 81, feature point feature amount 121 ₁ through 121 ₁₀ correspondingto the discriminators Cbn included in the shared information 65 ₂respectively.

Also, the lower side in FIG. 6 illustrates images for generation 141through 145 of which the feature points have been extracted at thefeature point extracting unit 24, and the feature amount has beencalculated at the feature amount extracting unit 25.

The whole feature amount calculating unit 26 calculates a correlationvalue between each of the feature point feature amount 121 ₁ through 121₁₀ corresponding to each of the discriminators Cbn included in theshared information 65 ₂ included in the shared code book 81, and each ofthe images for generation 141 through 145, as a whole feature amount.

Specifically, for example, the whole feature amount calculating unit 26takes a predetermined region (e.g., a rectangular region with theposition corresponding to the feature point of the feature point featureamount 121 _(n) as the center) based on the feature point correspondingto the feature point feature amount 121 _(n) (n is a natural number from1 to 10) of the whole region making up the image for generation 141, asa search range.

The whole feature amount calculating unit 26 calculates a correctionvalue between the feature point feature amount 121 _(n), and the featurepoint feature amount of each feature point existing on the search rangeof the image for generation 141, and takes the maximum correlation valueof the multiple correlation values obtained by calculation thereof, asthe final correlation value 161 _(n) between the feature point featureamount 121 _(n) and the image for generation 141.

The whole feature amount calculating unit 26 calculates correlationvalues 161 ₁ through 161 ₁₀ as to the feature point feature amount 121 ₁through 121 ₁₀ regarding the image for generation 141, and takes avector with the calculated correlation values 161 ₁ through 161 ₁₀ aselements, as the whole feature amount of the image for generation 141.

The whole feature amount calculating unit 26 calculates the wholefeature amount of each of the images for generation 142 through 145 inthe same way as with a case where the whole feature amount of the imagefor generation 141 has been calculated.

The whole feature amount calculating unit 26 supplies the whole featureamount of each of the multiple images for generation 141 through 145 tothe identifier generating unit 27.

Processing Performed by Identifier Generating Unit 27

Next, FIG. 7 illustrates the outline of processing to be performed bythe identifier generating unit 27.

The identifier generating unit 27 follows, for example, the Boostingalgorithm to select (the dimension of) dimensional feature amount to beused for identifying out of dimensional feature amount making up thewhole feature amount from the whole feature amount calculating unit 26,and also to generate an identifier for performing identification usingthe dimensional feature amount thereof.

Specifically, the identifier generating unit 27 generates, of theplurality of dimensional feature amounts (elements of a vector) makingup the whole feature amount from the whole feature amount calculatingunit 26 (FIG. 1), an identifier for performing identification usingdimensional feature amount that reduces an error value representing adegree of mistaking identification between a positive image and anegative image, and dimensional information representing the dimensionof dimensional feature amount that reduces the error value.

Specifically, now, let us say that there are multiple N images as imagesfor generation, and at the whole feature amount calculating unit 26, asillustrated in FIG. 7, a vector serving as the whole feature amount x₁,x₁, . . . , x_(N) of N samples of the N images for generation has beenobtained.

Further, the whole feature amount x_(i)(i=1, 2, . . . , N) is, asillustrated in FIG. 7, assumed to be an M dimensional vector havingmultiple M elements (dimensional feature amount) x_(1,1), x_(1,2), . . ., x_(i,M).

Also, as described in FIG. 1, the correct answer label is supplied fromthe whole feature amount calculating unit 26 to the identifiergenerating unit 27, but here, the correct answer label of the i'thsample x_(i) (i'th image for generation) is represented as y_(i). Thecorrect answer label y_(i) is assumed to be +1 in the event that thei'th image for generation is a positive image, and is assumed to be −1in the event that the i'th image for generation is a negative image.

The identifier that the identifier generating unit 27 generates is afunction for performing identification using dimensional feature amountx_(i,d) that reduces an error value representing a degree of mistakingidentification between a positive image and a negative image, of the Mpieces of the dimensional feature amount X_(i,1) through x_(i,M) makingup the whole feature amount x_(i), and is configured of multiple weaklearners h_(t,d)(x_(i,d)).

Here, let us say that the suffix t of the weak learners h_(t,d)(x_(i,d))is a variable for counting the number of the weak learnersh_(t,d)(x_(i,d)), and an identifier is made up of multiple T weaklearners h_(1,d)(x_(i,d)), h_(2,d)(x_(i,d)), . . . , h_(T,d)(x_(i,d)).

As for the number T of the weak learners h_(t,d)(x_(i,d)), a value equalto or smaller than M is set, for example, experientially, or so that theidentification rate of identification by an identifier becomes a certainamount of value.

The weak learners h_(t,d)(x_(i,d)) are functions for outputting anidentification result to the effect that the image for generation is apositive image or negative image with the d'th dimensional featureamount x_(i,d) of the whole feature amount x_(i) (the d'th element ofthe vector serving as the whole feature amount x_(i)) of the image forgeneration as input, and output, for example, +1 as an identificationresult to the effect that the image for generation is a positive image,and −1 as an identification result to the effect that the image forgeneration is a negative image, respectively.

Now, if we say that the error values of the identification results ofthe weak learners h_(t,d)(x_(i,d)) are represented as ε_(t,d), theidentifier generating unit 27 determines the weak learnersh_(t,d)(x_(i,d)) so that the error values ε_(t,d) become small.

Note that, in order to simplify description, as the weak learnersh_(t,d)(x_(i,d)), a function is assumed to be employed here wherein, forexample, in the event that the d'th dimensional feature amount x_(i,d)that is an argument is equal to or greater than a predeterminedthreshold, +1 is output, which represents the identification result tothe effect that the image for generation is a positive image, and in theevent that the d'th dimensional feature amount x_(i,d) that is anargument is less than a predetermined threshold, −1 is output, whichrepresents the identification result to the effect that the image forgeneration is a negative image.

In this case, determining the weak learners h_(t,d)(x_(i,d)) so that theerror values ε_(t,d) become small means to determine a threshold of theweak learners h_(t,d)(x_(i,d)). Of the N d'th dimensional feature amountx_(1,d), x_(2,d), . . . , x_(N,d) that can be served as arguments, avalue between the minimum value and the maximum value is determined asthe threshold of the weak learners h_(t,d)(x_(i,d)).

The identifier generating unit 27 determines each of the weak learnersh_(t,1)(x_(i,1)), h_(t,2)(x_(i,2)), . . . , h_(t,M)(x_(i,M)) so as toreduce each of the error values ε_(t,1), ε_(t,2), . . . , ε_(t,M), andobtains a dimension d(t) whereby the minimum value of the error valuesε_(t,1) through ε_(t,d) can be obtained (hereafter, referred to as“minimum error dimension”).

Also, the identifier generating unit 27 obtains weight D_(t)(i) to causean error of the identification result of the image for registration toaffect the error value ε_(t,d) for each image for generation dependingon whether the identification result after the i'th image for generationby the weak learners h_(t,d)(x_(i,d)) is matched with the correct answerlabel y_(i), i.e., whether Expression h_(t,d)(x_(i,d))=y_(i) holds orExpression h_(t,d)(x_(i,d))=y_(i) holds.

Here, the error value ε_(t,d) is obtained by adding the weight D_(t)(i)of an image for generation where the identification result by the weaklearners h_(t,d)(x_(i,d)) commits an error, of the N images forgeneration.

The identifier generating unit 27 repeats determining of the weaklearners h_(t,d)(x_(i,d)) so as to reduce the error value ε_(t,d),obtaining dimension (minimum error dimension) d(t) whereby the minimumvalue of the error values ε_(t,1) through ε_(t,M) of the identificationresult of the image for generation by the weak learnersh_(t,d)(x_(i,d)), and obtaining of weight D_(t)(i) to be used forcalculating the error value ε_(t,d) by T times, thereby generatingidentifier H(x) made up of the T weak learners h_(1,d)(x_(i,d)),h_(2,d)(x_(i,d)), . . . , h_(T,d)(x_(i,d)), and dimensional informationrepresenting the minimum error dimensions d(1), d(2), . . . , d(T).

Operational Description of Learning Device 1

Next, description will be made regarding learning processing to beperformed by the learning device 1 (hereafter, referred to as “firstlearning processing”), with reference to the flowchart in FIG. 8.

This first learning processing is started when an image for learning issupplied to the input unit 21. At this time, of an image for generation,a primitive image, and a model image, which are images for learning tobe supplied, the input unit 21 supplies the primitive image and modelimage to the shared code book generating unit 22, and supplies the imagefor generation to the feature point extracting unit 24.

In step S1, the shared code book generating unit 22 performs shared codebook generation processing for generating a shared code book 81 based onthe primitive image and model image from the input unit 21 to store thisin the shared code book storage unit 23. Note that the details of theshared code book generation processing will be described later withreference to the flowchart in FIG. 9.

In step S2, the feature point extracting unit 24 sequentially takes eachof the multiple images for generation from the input unit 21 as an imagefor generation of interest.

In step S3, the feature point extracting unit 24 extracts a featurepoint from the image for generation of interest to supply this to thefeature amount extracting unit 25 along with the image for generation ofinterest.

In step S4, the feature amount extracting unit 25 similarly extractsfeature point feature amount representing the feature of the featurepoint from the feature point extracting unit 24 from the image forgeneration of interest from the feature point extracting unit 24, andsupplies this to the whole feature amount calculating unit 26 in amanner correlated with the feature point from the feature pointextracting unit 24.

Also, the feature amount extracting unit 25 extracts a correct answerlabel added to the image for generation of interest from the featurepoint extracting unit 24, and supplies this to the whole feature amountcalculating unit 26.

In step S5, the whole feature amount calculating unit 26 calculates,based on the shared code book 81 stored in the shared code book storageunit 23, the whole feature amount representing the feature of the wholeof the image for generation of interest thereof from the feature pointfeature amount from the feature amount extracting unit 25.

Specifically, for example, as described with reference to FIG. 6, thewhole feature amount calculating unit 26 calculates the whole featureamount of the image for generation of interest based on the featurepoint feature amount correlated with each of the discriminators Cbnincluded in the shared information 65 (e.g., shared information 65 ₂ inthe event of generating an identifier for identifying whether or not anobject to be identified belongs to the category B of automobiles, anddimensional information) included in the shared code book 81.

The whole feature amount calculating unit 26 supplies the whole featureamount of the calculated image for generation of interest to theidentifier generating unit 27 along with the correct answer label fromthe feature amount extracting unit 25.

In step S6, the feature point extracting unit 24 determines whether ornot all of the multiple images for generation from the input unit 21have been taken as an image for generation of interest, and in the eventthat determination is made that all of the multiple images forgeneration from the input unit 21 have not been taken as an image forgeneration of interest, returns the processing to step S2.

Subsequently, in step S2, the feature point extracting unit 24 takes, ofthe multiple images for generation from the input unit 21, an image forgeneration that has not been taken as an image for generation ofinterest as a new image for generation of interest, and the processingproceeds to step S3, and thereafter, the same processing is performed.

Also, in the event that determination is made in step S6 that all of themultiple images for generation from the input unit 21 have been taken asan image for generation of interest, the processing proceeds to step S7.

In step S7, the identifier generating unit 27 performs identifiergeneration processing for generating an identifier and dimensionalinformation using the whole feature amount of each of the multipleimages for generation, and the correct answer label of each of themultiple images for generation, in response to the corresponding wholefeature amount and correct answer label being supplied regarding each ofthe multiple images for generation from the whole feature amountcalculating unit 26. This is the end of the first learning processing.Note that the details of the first learning processing will be describedlater with reference to the flowchart in FIG. 11.

Details of Shared Code Book Generation Processing

Next, the details of the shared code book generation processing in stepS1 in FIG. 8 will be described with reference to the flowchart in FIG.9.

In step S21, the feature point extracting unit 41 through the code bookgenerating unit 43 of the shared code book generating unit 22 performcode book generation processing for generating a code book in which thefeature point feature amount extracted from the multiple primitiveimages from the input unit 21 is correlated with the discriminator Cbfor uniquely identifying the feature point feature amount thereof, andsupplying and storing this in the code book storage unit 44. Note thatthe details of the code book generation processing will be describedwith reference to the flowchart in FIG. 10.

In step S22, the feature point extracting unit 45 obtains, of themultiple model images from the input unit 21, a model image where anobject to be identified belonging to a predetermined category (e.g.,notebook personal computer) exists.

In step S23, the feature point extracting unit 45 sequentially takeseach of the model images obtained in the processing in step S22 as amodel image of interest.

In step S24, the feature point extracting unit 45 extracts a featurepoint from the model image of interest, and supplies this to the featureamount extracting unit 46 along with the model image of interest.

In step S25, the feature amount extracting unit 46 similarly extractsfeature point feature amount representing the feature of the featurepoint from the feature point extracting unit 45 from the model image ofinterest from the feature point extracting unit 45. Subsequently, thefeature amount extracting unit 46 correlates the extracted feature pointfeature amount with the feature point from the feature point extractingunit 45.

In step S26, the feature point extracting unit 45 determines whether ornot all of the model images obtained in the processing in step S22 havebeen taken as a model image of interest, and in the event that all ofthe model images obtained in the processing in step S22 have not beentaken as a model image of interest yet, returns the processing to stepS23.

Subsequently, in step S23, the feature point extracting unit 45 takes amodel image that has not been taken as a model image of interest of themodel images obtained in the processing in step S22 as a new model imageof interest, advances the processing to step S24, and thereafter, thesame processing is performed.

Also, in the event that determination is made in step S26 that all ofthe model images obtained in the processing in step S22 have been takenas a model image of interest, the feature point extracting unit 45informs the feature amount extracting unit 46 accordingly.

In response to the notice from the feature point extracting unit 45, thefeature amount extracting unit 46 generates an initial feature amountpool 62 including the feature point feature amount extracted in theprocessing in step S25, supplies this to the feature amount replacingunit 47, and advances the processing to step S27.

In step S27, the feature amount replacing unit 47 replaces the featurepoint feature amount included in the initial feature amount pool 62 fromthe feature amount extracting unit 46 with the discriminator Cbnincluded in the code book 61 stored in the code book storage unit 44.

Specifically, for example, the feature amount replacing unit 47 readsout the code book 61 from the code book storage unit 44. Subsequently,the feature amount replacing unit 47 replaces the feature point featureamount fvec′ included in the initial feature amount pool 62 from thefeature amount extracting unit 46 with the discriminator Cbn foridentifying the feature point feature amount fvec most similar to thefeature point feature amount fvec′ of the plurality of feature pointfeature amounts included in the read code book 61.

In this way, the feature amount replacing unit 47 replaces each of theplurality of feature point feature amounts included in the initialfeature amount pool 62 from the feature amount extracting unit 46 withthe discriminator Cbn included in the code book 61 stored in the codebook storage unit 44.

Thus, the feature amount replacing unit 47 generates a feature amountpool 63 including the discriminator Cbn correlated with the featurepoint from the initial feature amount pool 62 including the featurepoint feature amount fvec′ correlated with the feature point. Thefeature amount replacing unit 47 supplies the generated feature amountpool 63 to the quantizing unit 48.

Also, the feature amount replacing unit 47 supplies the code book 61read out from the code book storage unit 44 to the quantizing unit 48.

In step S28, the quantizing unit 48 quantizes the feature pointcorrelated with the discriminator Cbn included in the feature amountpool 63 from the feature amount replacing unit 47, and obtains aquantized feature amount pool 64 obtained as a result thereof.

Subsequently, the quantizing unit 48 obtains shared information 65 to beobtained by grouping the same discriminators Cbn of the discriminatorsCbn included in the obtained quantized feature amount pool 64.

In step S29, the feature point extracting unit 45 determines whether ornot, of the multiple model images from the input unit 21, the modelimages regarding all of the categories have been obtained in theprocessing in step S22, and in the event that determination is made thatthe model images regarding all of the categories have not been obtained,returns the processing to step S22.

Subsequently, in step S22, the feature point extracting unit 45 obtains,of the multiple model images from the input unit 21, the model image ofan object to be identified belonging to a category that has not beenobtained yet (e.g., automobiles), advances the processing to step S23,and thereafter, the same processing is performed.

Also, in the event that determination is made in step S29 that of themultiple model images from the input unit 21, the model images regardingall of the categories have been obtained, the feature point extractingunit 45 advances the processing to step S30.

In step S30, the quantizing unit 48 supplies and stores, for eachcategory, the shared information 65 (e.g., the shared information 65 ₁of the category A, the shared information 65 ₂ of the category B, etc.)obtained in the processing in step S28, and the code book 61 from thefeature amount replacing unit 47 to the shared code book storage unit 23as a shared code book 81, and returns the processing to step S1 in FIG.8.

Details of Code Book Generation Processing

Next, description will be made regarding the details of the code bookgeneration processing for generating a code book 61 to be performed bythe feature point extracting unit 41 through the code book generatingunit 43 of the shared code book generating unit 22 in step S21 in FIG.9, with reference to the flowchart in FIG. 10.

In step S51, the feature point extracting unit 41 sequentially takeseach of the multiple primitive images from the input unit 21 as aprimitive image of interest.

In step S52, the feature point extracting unit 41 extracts a featurepoint from the primitive image of interest, and supplies this to thefeature amount extracting unit 42 along with the primitive image ofinterest.

In step S53, the feature amount extracting unit 42 similarly extractsfeature point feature amount representing the feature of the featurepoint from the feature point extracting unit 41 from the primitive imagefrom the feature point extracting unit 41, and supplies this to the codebook generating unit 43.

In step S54, the feature point extracting unit 41 determines whether ornot all of the multiple primitive images from the input unit 21 havebeen taken as a primitive image of interest, and in the event thatdetermination is made that all of the multiple primitive images from theinput unit 21 have not been taken as a primitive image of interest yet,returns the processing to step S51.

Subsequently, in step S51, the feature point extracting unit 41 takes,of the multiple primitive images from the input unit 21, a primitiveimage that has not been taken as a primitive image of interest yet, as anew primitive image of interest, advances the processing to step S52,and thereafter, the same processing is performed.

Also, in the event that determination is made in step S54 that all ofthe multiple primitive images from the input unit 21 have been taken asa primitive image of interest, the feature point extracting unit 41advances the processing to step S55.

In step S55, the code book generating unit 43 generates a code book madeup of feature point feature amount representing each group obtained bygrouping the plurality of feature point feature amounts, based on theplurality of feature point feature amounts supplied from the featureamount extracting unit 42.

Specifically, for example, the code book generating unit 43 groups theplurality of feature point feature amounts to be supplied from thefeature amount extracting unit 42 into several groups by a clusteringmethod such as the k-means method in the feature amount space.Subsequently, the code book generating unit 43 determines, for everyseveral groups obtained by grouping, the centroid of feature pointfeature amount belonging to the groups thereof as feature point featureamount representing the groups.

Also, the code book generating unit 43 correlates the feature pointfeature amount determined for every several groups with thediscriminator Cbn for uniquely identifying the feature point featureamount thereof. Also, the code book generating unit 43 generates a codebook 61 including the feature point feature amount correlated with thediscriminator Cbn, supplies and stores this to the code book storageunit 44, and return the processing to step S21 in FIG. 9.

Details of Identifier Generation Processing

Next, description will be made regarding the details of the identifiergeneration processing for generating an identifier and dimensionalinformation to be performed by the identifier generating unit 27 in stepS7 of FIG. 8, with reference to the flowchart in FIG. 11.

In step S71, the identifier generating unit 27 sets the initial valuesD₁(1), D₁(2), . . . , D₁ (N) of the weight D_(t)(i) to cause an error ofthe identification result of the image for registration to affect theerror value ε_(t,d) representing a degree of the weak learnersh_(t,d)(x_(i,d)) mistaking identification, for example, in accordancewith the following Expression (2), and advances the processing to stepS72.

$\begin{matrix}{{D_{t}(i)} = \frac{1}{N}} & (2)\end{matrix}$

In step S72, the identifier generating unit 27 initializes a variable tfor counting the number of the weak learners h_(t,d)(x_(i,d)) making upthe identifier H(x) to 1, and advances the processing to step S73.

In step S73, the identifier generating unit 27 determines (the thresholdTH_(t,d)) of the weak learners h_(t,d)(x_(i,d)) so that the error valueε_(t,d) to be obtained by using the weight D_(t)(i) becomes the minimum,regarding each of the dimensions d=1, 2, . . . , M of the whole featureamount x_(i), and advances the processing to step S74.

Here, in step S73, the identifier generating unit 27 determines thethreshold TH_(t,d) of the weak learners h_(t,d)(x_(i,d)) so that theerror value ε_(t,d) to be calculated in accordance with the followingExpression (3) becomes the minimum for example.

$\begin{matrix}{ɛ_{t,d} = {\sum\limits_{i = t}^{N}{{D_{t}(i)}\left\lbrack {y_{i} \neq {h_{t,d}\left( x_{i,d} \right)}} \right\rbrack}}} & (3)\end{matrix}$

In Expression (3), [y_(i)≠h_(t,d)(x_(i,d))] is an indicator function, 1is obtained when the expression y_(i)≠h_(t,d)(x_(1,d)) holds, and 0 isobtained when the expression y_(i)≠h_(t,d)(x_(1,d)) does not hold.

Therefore, according to Expression (3), the error value ε_(t,d) isobtained by adding only the weight D_(t)(i) of the image for generationwhere the identification result by the weak learners h_(t,d)(x_(i,d))mistakes (image for generation where the expressiony_(i)≠h_(t,d)(x_(i,d)) holds) of the N images for generation.

In step S74, the identifier generating unit 27 uses the weak learnersh_(t,d)(x_(i,d)) determined regarding each of the dimensions d=1, 2, . .. , M in the last processing in step S73 to obtain the minimum valueε_(t) out of the error values ε_(t,1), ε_(t,2), . . . , ε_(t,M) to becalculated in accordance with Expression (3). Further, the identifiergenerating unit 27 obtains the dimension (minimum error dimension) d(t)(an integer value in a range of 1 through M) whereby the minimum valueε_(t) of the error values ε_(t,1) through ε_(t,M) can be obtained, andadvances the processing to step S75.

Here, the minimum error dimension d(t) is the dimension of dimensionalfeature amount to be used for identification by the identifier H(x) ofthe dimensional feature amount making up the whole feature amount.Accordingly, with identification by the identifier H(x), the dimensionalfeature amount of the minimum error dimension d(t) is selected out ofthe dimensional feature amount making up the whole feature amount, andis used for identification.

Also, if we say that the minimum value ε_(t) of the error valuesε_(t,1), ε_(t,2), . . . , ε_(t,M) is the minimum error value ε_(t), theweak learners h_(t,d)(t)(x_(i,d)(t)) whereby the minimum error valueε_(t) thereof can be obtained becomes the t'th weak learner making upthe identifier H(x).

In step S75, the identifier generating unit 27 uses the minimum errorvalue ε_(t) obtained in the last processing in step S74 to obtainreliability a representing the reliability of identification of theimage for generation by the t'th weak learner h_(t,d)(t)(x_(i,d)(t))making up the identifier H(x) in accordance with the followingExpression (4), and advances the processing to step S76.

$\begin{matrix}{\alpha_{t} = {\frac{1}{2}{\ln\left( \frac{1 - ɛ_{t}}{ɛ_{t}} \right)}}} & (4)\end{matrix}$

Here, in Expression (4), In represents a natural logarithm, andaccording to Expression (4), the greater (or smaller) the minimum errorvalue ε_(t) is, the smaller (or greater) the value of the reliabilityα_(t) is obtained.

In step S76, the identifier generating unit 27 updates the weightD_(t)(i) to weight D_(t+1)(i) in accordance with the followingExpression (5), and advances the processing to step S77.

$\begin{matrix}\begin{matrix}{{D_{t + 1}(i)} = {\frac{D_{t}(i)}{Z_{t}} \times \left\{ \begin{matrix}{\mathbb{e}}^{- \alpha_{t}} & {{{if}\mspace{14mu}{h_{t,{d{(t)}}}\left( x_{i,{d{(t)}}} \right)}} = y_{i}} \\{\mathbb{e}}^{\alpha_{i}} & {{{if}\mspace{14mu}{h_{t,{d{(t)}}}\left( x_{i,{d{(t)}}} \right)}} \neq y_{i}}\end{matrix} \right.}} \\{= {\frac{D_{t}(i)}{Z_{t}} \times {\mathbb{e}}^{{- \alpha_{t}}y_{i}{h_{t,{d{(t)}}}{(x_{1,{d{(t)}}})}}}}}\end{matrix} & (5)\end{matrix}$

Here, in Expression (5), a coefficient Z_(t) is a coefficient fornormalization of the weight D_(t+1)(i), and is represented with thefollowing Expression (6).

$\begin{matrix}{Z_{t} = {\sum\limits_{i = 1}^{N}{{D_{1}(i)}{\mathbb{e}}^{{- \alpha_{t}}y_{i}{h_{t,{d{(t)}}}{(x_{i,{d{(t)}}})}}}}}} & (6)\end{matrix}$

According to Expression (6), with regard to the i'th image forgeneration where the identification result by the weak learnerh_(t,d)(t)(x_(i,d)(t)) is correct, i.e., the image for generation wherethe identification result is matched with the correct answer labely_(i), the weight D_(t)(i) is updated to weight D_(t+1)(i) having asmaller value. As a result thereof, in the next step S73, the errorvalue ε_(t,d) to be calculated by using the weight D_(t)(i) becomes asmaller value.

On the other hand, with regard to the i'th image for generation wherethe identification result by the weak learner h_(t,d)(t)(x_(i,d)(t)) isincorrect, i.e., the image for generation where the identificationresult is not matched with the correct answer label y_(i), the weightD_(t)(i) is updated to weight D_(t+1)(i) having a greater value. As aresult thereof, in the next step S73, the error value ε_(t,d) to becalculated by using the weight D_(t)(i) becomes a greater value.

In step S77, the identifier generating unit 27 determines whether or notthe variable t is equal to the number T of the weak learnersh_(t,d)(x_(i,d)) making up the identifier H(x).

In the event that determination is made in step S77 that the variable tis unequal to the number T of the weak learners, the identifiergenerating unit 27 advances the processing to step S78, increments thevariable t by one, returns the processing to step S73 from step S78, andthereafter, the same processing is performed.

Also, in the event that determination is made in step S77 that thevariable t is equal to the number T of the weak learners, i.e., in theevent that the T weak learners h_(1,d)(1)(x_(i,d) (1)), h_(2,d)(2)(x_(i,d) (2)), . . . , h_(T,d)(T) (x_(i,d)(T)) making up the identifierH(x) and the T minimum error dimensions d(1), d(2), . . . , d(T) havebeen generated, the identifier generating unit 27 advances theprocessing to step S79.

In step S79, the identifier generating unit 27 outputs the T weaklearners h_(i,d)(1) (x_(i,d)(1)), h_(2,d)(2)(x_(1,d)(2)), . . . ,h_(T,d)(T)(x_(i,d)(T)), and the T pieces of reliability α₁, α₂, . . . ,α_(T) as (the parameters stipulating) the identifier H(x).

Further, in step S79, the identifier generating unit 27 outputs the Tminimum error dimensions d(1), d(2), . . . , d(T) as dimensionalinformation, ends the identifier generation processing, and returns theprocessing to step S7 in FIG. 8.

With the identifier generating unit 27, as described above, according tothe statistic learning by boosting (first learning processing), thereare obtained, of the dimensional feature amount making up the wholefeature amount, the dimensions (minimum error dimensions) d(1) throughd(T) representing the T dimensional feature amount effective foridentifying an object to be identified, and the identifier H(x) forperforming identification using the dimensional feature amount of theminimum error dimension d(t).

In this way, with the first learning processing to be performed by thelearning device 1, in the event of generating the minimum errordimensions d(1) through d(T) serving as dimensional information, and theidentifier H(x), the shared code book 81 is employed, which shares thefeature point feature amount included in the code book 61 (e.g., thefeature point feature amount fvec to be identified by the discriminatorCb₃₀) between different categories to be identified (e.g., categories Aand B).

Therefore, with the first learning processing, as compared to a casewhere an identifier and so forth are generated by using a code bookgenerated for every category, multiple identifiers for identifying eachof different objects to be identified can be generated withoutincreasing the feature point feature amount included in the code book 61included in the shared code book 81.

Also, with the first learning processing, the shared information 65included in the shared code book 61 holds a feature point extracted froma model image for every category. Specifically, for example, the sharedinformation 65 ₁ holds a feature point extracted from a model imagewhere a notebook personal computer belonging to the category A exists,and the shared information 65 ₂ holds a feature point extracted from amodel image where an automobile belonging to the category B exists,respectively.

Here, a feature point extracted from a model image where a notebookpersonal computer belonging to the category A exists generally differsfrom a feature point extracted from a model image where an automobilebelonging to the category B exists.

Accordingly, the shared information 65 included in the shared code book61 holds a different feature point for every category.

Therefore, in the event that in step S5 of the first learningprocessing, the whole feature amount calculating unit 26 calculates thewhole feature amount, a position where the search region of the imagefor generation exists, which is determined based on the feature pointheld in the shared information 65, differs for every category to beidentified.

Therefore, with the whole feature amount calculating unit 26, forexample, as compared to a case where the whole region on the image forgeneration is divided into grid-shaped search ranges, and a search rangecommon between categories (search range existing one the same position)is employed, the whole feature amount to be used for generation of anidentifier can be calculated in a more accurate manner.

Accordingly, in step S7 of the first learning processing, the identifiergenerating unit 27 can generate an identifier (and dimensionalinformation) with higher precision based on the calculated whole featureamount.

Configuration Example of Identifying Device 181

FIG. 12 illustrates a configuration example of an identifying device 181for performing identification using the identifier H(x) and thedimensional information d(1), d(2), . . . , d(T) obtained by thelearning device 1.

This identifying device 181 uses the identifier H(x) obtained by thelearning device 1, and the minimum error dimensions d(1) through d(T)serving as dimensional information to identify whether or not a subjectexisting on an image to be processed is a predetermined object to beidentified, and specifically, whether or not a subject existing on animage to be processed belongs to a predetermined category.

Specifically, this identifying device 181 is configured of a shared codebook storage unit 201, a dimensional information storage unit 202, anidentifier storage unit 203, a feature point extracting unit 204, afeature amount extracting unit 205, a whole feature amount calculatingunit 206, and an identifying unit 207.

The shared code book storage unit 201 stores the same shared code book81 as that stored in the shared code book storage unit 23 (FIG. 1) ofthe learning device 1 beforehand.

The dimensional information storage unit 202 stores the minimum errordimensions d(1) through d(T) beforehand serving as the dimensionalinformation obtained regarding a predetermined object to be identifiedat the identifier generating unit 27 of the learning device 1.

The identifier storage unit 203 stores the T weak learners h_(1,d)(1)(x_(i,d)(1)), h_(2,d)(2)(x_(i,d)(2)), . . . , h_(T,d)(T) (x_(i,d)(T))serving as the identifier H(x) obtained regarding a predetermined objectto be identified at the identifier generating unit 27 of the learningdevice 1, and the T pieces of reliability α₁, α₂, . . . , α_(T)beforehand.

An image to be processed that is an object to be identified regardingwhether or not a subject existing on the image is an object to beidentified is supplied to the feature point extracting unit 204. Thefeature point extracting unit 204 extracts a feature point from thesupplied image to be processed in the same way as with the feature pointextracting unit 24 of the learning device 1, and supplies this to thefeature amount extracting unit 205 along with the image to be processed.

The feature amount extracting unit 205 similarly extracts the featurepoint feature amount of the feature point from the feature pointextracting unit 204 from the image to be processed from the featurepoint extracting unit 204 in the same way as with the feature amountextracting unit 25 of the learning device 1.

Subsequently, the feature amount extracting unit 205 supplies theextracted feature point feature amount to the whole feature amountcalculating unit 206 in a manner correlated with the feature point fromthe feature point extracting unit 204.

The whole feature amount calculating unit 206 obtains, in the same wayas with the whole feature amount calculating unit 26 of the learningdevice 1, from the feature point feature amount of the image to beprocessed from the feature amount extracting unit 205, dimensionalfeature amount making up the whole feature amount of the image to beprocessed thereof, based on the feature point feature amountcorresponding to each of the discriminators Cbn included in the sharedinformation 65 (e.g., the shared information 65 ₂ of the category B inthe event of identifying whether or not an object to be identifiedbelongs to the category B of automobiles) included in the shared codebook 81 stored in the shared code book storage unit 201.

However, with the whole feature amount calculating unit 206, not all ofthe M pieces of the dimensional feature amount making up the wholefeature amount of the image to be processed but the dimensional featureamount of the minimum error dimensions d(1) through d(T) serving as thedimensional information stored in the dimensional storage unit 202 ofthe M pieces of the dimensional feature amount thereof is selectivelyobtained.

Note that, with the whole feature amount calculating unit 206, from thestart, of the whole feature amount of the image to be processed, onlythe dimensional feature amount of the minimum error dimensions d(1)through d(T) serving as the dimensional information may be obtained, orafter the whole feature amount of the image to be processed is obtained,the dimensional feature amount of the minimum error dimensions d(1)through d(T) may be obtained out of the whole feature amount thereof.

Now, for example, let us say that a vector with M pieces of dimensionalfeature amount as elements, which is configured of M pieces ofdimensional feature amount, and serves as the whole feature amount ofthe image to be processed, is represented as x′. Also, the m'th of the Mpieces of the dimensional feature amount of the whole feature amount x′of the image to be processed is represented as x_(m)′.

In this case, of the M pieces of the dimensional feature amount of thewhole feature amount x′ of the image to be processed, the dimensionalfeature amount of the minimum error dimensions d(1) through d(T) isrepresented as x_(d(1))′, x_(d(2))′, . . . , x_(d(T))′.

The whole feature amount calculating unit 206 selects (selectivelyobtains), of the M pieces of the dimensional feature amount of the wholefeature amount x′ of the image to be processed, the T pieces of thedimensional feature amount x_(d(1))′ through x_(d(T))′ of the minimumerror dimensions d(1) through d(T), and supplies these to theidentifying unit 207.

The identifying unit 207 identifies whether or not a subject existing onthe image to be processed is a predetermined object to be identified, byproviding the dimensional feature amount x_(d(1))′ through x_(d(T))′ ofthe minimum error dimensions d(1) through d(T) of the image to beprocessed to the identifier H(x′) stored in the identifier storage unit203 as input x′, and outputs the identification result thereof.

Specifically, the identifying unit 207 uses the T weak learnersh_(1,d)(1)(x_(d(1))′), h_(2,d)(2)(x_(d(2))′), . . . ,h_(T,d)(T)(x_(d(T))′), and the T pieces of reliability α₁, α₂, . . . ,α_(T) as the identifier H(x′) stored in the identifier storage unit 203to calculate the function H(x′) of the following Expression (7) servingas the identifier H(x′).

$\begin{matrix}{{H\left( x^{\prime} \right)} = {{sign}\left( {\sum\limits_{t = 1}^{T}{\alpha_{t}{h_{t,{d{(t)}}}\left( x_{d{(t)}}^{\prime} \right)}}} \right)}} & (7)\end{matrix}$

Here, in Expression (7), sign( ) is a function to output, for example,+1 when a sign within parentheses is positive, and −1 when the sign isnegative, respectively. Accordingly, the value of the function H(x′) inExpression (7) becomes +1 or −1.

In the event that the value of the function H(x′) in Expression (7) is+1, this represents an identification result to the effect that asubject exiting on the image to be processed is a predetermined objectto be identified, and in the event that the value of the function H(x′)in Expression (7) is −1, this represents an identification result to theeffect that a subject exiting on the image to be processed is not apredetermined object to be identified.

Operational Description of Identifying Device 181

Next, description will be made regarding identification processing(hereafter, referred to as “first identification processing”) to beperformed by the identifying device 181 with reference to the flowchartin FIG. 13.

This first identification processing is started, for example, in theevent that the image to be processed has been supplied to the featurepoint extracting unit 204 of the identifying device 181.

In step S91, the feature point extracting unit 204 extracts a featurepoint from the supplied image to be processed, supplies this to thefeature amount extracting unit 205 along with the image to be processed,and advances the processing to step S92.

In step S92, the feature amount extracting unit 205 similarly extractsthe feature point feature amount of the feature point from the featurepoint extracting unit 204 from the image to be processed, supplied fromthe feature point extracting unit 204, supplies this to the wholefeature amount calculating unit 206, and advances the processing to stepS93.

In step S93, the whole feature amount calculating unit 206 obtains fromthe feature point feature amount of the image to be processed, suppliedfrom the feature amount extracting unit 205, of the dimensional featureamount making up the whole feature amount of the image to be processedthereof, the dimensional feature amount x_(d(1))′ through x_(d(T))′ ofthe minimum error dimensions d(1) through d(T) serving as thedimensional information stored in the dimensional information storageunit 202, based on the shared code book 81 stored in the shared codebook storage unit 201.

Specifically, for example, the whole feature amount calculating unit 206obtains the dimensional feature amount x_(d(1))′ through x_(d(T))′ ofthe minimum error dimensions d(1) through d(T) based on the featurepoint feature amount correlated with each of the discriminators Cbnincluded in the shared information 65 (e.g., shared information 65 ₂ inthe event of identifying whether an object to be identified belongs tothe category B of mobiles) included in the shared code book 81.

Subsequently, the whole feature amount calculating unit 206 supplies thedimensional feature amount x_(d(1))′ through x_(d(T))′ of the minimumerror dimensions d(1) through d(T) to the identifying unit 207, andadvances the processing to step S94.

In step S94, the identifying unit 207 identifies whether or not asubject existing on the image to be processed is a predetermined objectto be identified, by providing the dimensional feature amount x_(d(1))′through x_(d(T))′ of the minimum error dimensions d(1) through d(T) ofthe image to be processed to the identifier H(x′) stored in theidentifier storage unit 203 and represented with Expression (7) as inputx′, outputs the identification result thereof, and the firstidentification processing is ended.

As described above, with the first identification processing to beperformed by the identifying device 181, in the event that the wholefeature amount calculating unit 206 obtains the dimensional featureamount x_(d(1))′ through x_(d(T))′ in step S93, the shared code book 81is employed wherein the feature point feature amount (e.g., featurepoint feature amount fvec to be identified by the discriminator Cb₃₀)included in the code book 61 is shared between different categories tobe identified (e.g., categories A and B).

Accordingly, for example, with the identifying device 181, even in theevent that there are multiple categories to be identified, unlike a casewhere a different code book is generated for every multiple categories,increase in feature point feature amount included in the code book 61included in the shared code book 81 can be prevented.

Also, in step S93 of the first identification processing, the wholefeature amount calculating unit 206 calculates, in the same way as withthe whole feature amount calculating unit 26 in step S5 of the firstlearning processing, the dimensional feature amount x_(d(1))′ throughx_(d(T))′ of the minimum error dimensions d(1) through d(T) of thedimensional feature amount making up the whole feature amount of theimage to be processed.

Therefore, with the whole feature amount calculating unit 206, in thesame way as with the whole feature amount calculating unit 26, forexample, as compared to a case where the whole region on the image to beprocessed is divided into grid-shaped search ranges, and a common searchrange (search range existing in the same position) is used betweencategories, the dimensional feature amount x_(d(1))′ through x_(d(T))′of the minimum error dimensions d(1) through d(T) of the dimensionalfeature amount making up the whole feature amount of the image to beprocessed, can be calculated in a more accurate manner.

Accordingly, in step S94 of the first identification processing, withthe identifying unit 207, as compared to a case where a common searchrange is used between categories to be identified, based on thedimensional feature amount x_(d(1))′ through x_(d(T))′ calculated by thewhole feature amount calculating unit 206, an object to be identifiedcan be identified in a more accurate manner.

With the first embodiment, at the whole feature amount calculating unit26 of the learning device 1, in the event of calculating a correlationvalue as an element (dimensional feature amount) of the whole featureamount, the whole feature amount of the image for generation iscalculated in the same size of a search range as to any object to beidentified regardless of the type of the object to be identified, but itis desirable for the size of the search range to be set to differentsizes according to the objects to be identified.

Specifically, for example, it can be conceived that in the event of anobject to be identified of which the shape has already been determined,such as an automobile, a relatively small search range is employed, andin the event of an object to be identified of which the shape has notbeen determined, such as an animal, a relatively great search range isemployed.

Also, in the event that the object to be identified is a roof existingin a relatively high place, it can be conceived that the search range isset to be great in the upward direction with a feature point as areference.

In this way, if a suitable search range is determined for each object tobe identified, the whole feature amount of the image for generation canbe calculated in a more accurate manner, whereby the precision ofidentification of an identifier to be generated based on the wholefeature amount can be improved.

Second Embodiment Configuration Example of Learning Device 221

Next, FIG. 14 describes regarding a learning device 221 for determininga search range for each feature point included in the shared information65 included in the shared code book 81 to calculate the whole featureamount. FIG. 14 illustrates a configuration example of the learningdevice 221 to which the second embodiment of the present invention hasbeen applied.

Note that, with this learning device 221, portions configured in thesame way as those of the learning device 1 in FIG. 1 according to thefirst embodiment are denoted with the same reference numerals, sodescription thereof will be omitted hereafter.

Specifically, with the learning device 221, a range determining unit 241is newly provided, and also a shared code book storage unit 242 and awhole feature amount calculating unit 243 are provided instead of theshared code book storage unit 23 and whole feature amount calculatingunit 26, but the others are configured in the same way as those in thelearning device 1.

The image for generation is provided from the input unit 21 to the rangedetermining unit 241. The range determining unit 241 reads out theshared code book 81 stored in the shared code book storage unit 242.

Subsequently, the range determining unit 241 performs rangedetermination processing for determining a search range representing arange on the image for generation to be searched for obtaining acorrelation value as to the image for generation from the input unit 21for each feature point included in the shared information 65 included inthe shared code book 81. Note that the details of the rangedetermination processing to be performed by the range determining unit241 will be described later with reference to FIG. 15.

The range determining unit 241 generates a shared code book 81′ in whichthe determined search range is correlated with a feature point includedin the shared information 65 included in the shared code book 81 basedon the shared code book 81, and supplies and stores this in the sharedcode book storage unit 242.

The shared code storage unit 242 stores the shared code book 81 from theshared code book generating unit 22. Also, the shared code book storageunit 242 stores the shared code book 81′ generated and supplied from therange determining unit 241.

Note that an arrangement may be made wherein the shared code bookgenerating unit 22 supplies the generated shared code book 81 not to theshared code book storage unit 242 but to the range determining unit 241,and the range determining unit 241 generates a shared code book 81′based on the shared code book 81 from the shared code book generatingunit 22, and supplies and stores this in the shared code book storageunit 242.

In this case, the shared code book storage unit 242 stores only theshared code book 81′ from the range determining unit 241.

The correct answer label of the image for generation of interest, andthe feature point feature amount extracted from the image for generationof interest are supplied to the whole feature amount calculating unit243 in the same way as with the whole feature amount calculating unit26.

The whole feature amount calculating unit 243 calculates whole featureamount representing the feature of the whole of the image for generationof interest thereof from the feature point feature amount from thefeature amount extracting unit 25 based on the shared code book 81′stored in the shared code book storage unit 242. Note that with regardto the processing to be performed by the whole feature amountcalculating unit 243 will be described later with reference to FIG. 16.

Subsequently, the whole feature amount calculating unit 243 supplies thecalculated whole feature amount of the image for generation of interestto the identifier generating unit 27 along with the correct answer labelof the image for generation of interest from the feature amountextracting unit 25.

Range Determination Processing to be Performed by Range Determining Unit241

FIG. 15 illustrates an example of the range determination processing tobe performed by the range determining unit 241.

The left side in FIG. 15 illustrates the shared information 65 includedin the shared code book 81, and the images for generation 261 ₁ through261 ₅. Note that of the images for generation 261 ₁ through 261 ₅, theimages for generation 261 ₁ through 261 ₃ represent positive images, andthe images for generation 261 ₄ and 261 ₅ represent negative images.

The middle in FIG. 15 illustrates correlation values x_(i) (Corr_1through Corr_5) serving as predetermined dimensional feature amount tobe calculated from each of the images for generation 261 ₁ through 261₅, and the correct answer label y_(i) (y_(i) is a value of either +1 or−1) of each of the images for generation 261 ₁ through 261 ₅.

The right side in FIG. 15 illustrates an error map 281 holding an errorrate Error (illustrated with a rectangle) representing a degree ofmistaking identification regarding whether each of the images forgeneration 261 ₁ through 261 ₅ is a positive image or negative image foreach candidate range that is a candidate of the search range.

Note that, with the error map 281, a horizontal axis Search x representsthe length in the horizontal direction (x direction) of the candidaterange, and a vertical axis Search y represents the length in thevertical direction (y direction) of the candidate range.

Accordingly, with the error map 281, of the multiple rectangles, forexample, the rectangle 301 illustrates an error rate Error in thecandidate range of 10×10 pixels (width×height).

Here, the error rate Error is matched with the above error value in thatthe error rate Error is error information representing a degree ofmistaking identification of an image for generation (positive image ornegative image).

However, the error value is used for generating an identifier anddimensional information, and differs from the error rate Error in thatthe error rate Error is used for determining a search range out of themultiple candidate ranges, so with the present Specification, the errorrate Error and the error value are distinctively described.

The range determining unit 241 reads out the shared code book 81 storedin the shared code book storage unit 242 from the shared code bookstorage unit 242.

The range determining unit 241 sequentially takes multiple featurepoints included in each piece of the shared information 65 included inthe read shared code book 81 (e.g., shared information 65 ₁ and 65 ₂) asa feature point of interest.

Subsequently, the range determining unit 241 calculates an error rateError for every multiple candidate ranges that are candidates of thesearch range of the feature point of interest.

Specifically, for example, the search determining unit 241 calculates,with regard to an image for generation 261 ₁, a correlation valuex_(i)(=Corr_1) with the feature point feature amount corresponding tothe feature point of interest in a candidate range 261S₁ of 10×10 pixels(width×height) with a position 261C₁ on an image for generation 261 ₁corresponding to the feature point of interest.

The range determining unit 241 calculates, in the same way as with thecase of the image for generation 261 ₁, correlation values x_(i)(=Corr_2through Corr_5) with the feature point feature amount corresponding tothe feature point of interest regarding each of the images forgeneration 261 ₂ through 261 ₅.

Subsequently, the range determining unit 241 calculates an error rate inthe search range of 10×10 pixels using the following Expression (8)based on the correlation values x_(i)(=Corr_1 through Corr_5) calculatedregarding each of the images for generation 261 ₁ through 261 ₅, and thecorrect answer label y_(i) of each of the images for generation 261 ₁through 261 ₅.Error=Ew[1((y _(i) ≠f(x _(i)))]  (8)

Note that in Expression (8), f(x_(i)) is a function for identifyingwhether each of the images for generation 261 ₁ through 261 ₅ is apositive image or negative image based on the correlation value x_(i).

The function f(x_(i)) outputs a value 1 representing that thecorresponding image for generation has been identified as a positiveimage in the event that the correlation value x_(i) is equal to orgreater than a threshold Thresh, and outputs a value −1 representingthat the corresponding image for generation has been identified as anegative image in the event that the correlation value x_(i) is lessthan a threshold Thresh.

Note that the threshold Thresh is taken as a value between the minimumvalue and the maximum value of the correlation values x_(i), and istaken as a value whereby difference between the correct answer labely_(i), and the value of the corresponding function f(x_(i)) becomes theminimum, i.e., a value whereby the error rate Error becomes the minimum.

Specifically, for example, each of the calculated correlation valuesx_(i) (=Corr_1 through Corr_5) is taken as a candidate threshold servingas a candidate of the threshold Thresh, and the error rate Error byExpression (8) is calculated. Subsequently, of the multiple candidatethresholds, a candidate threshold whereby difference between the correctanswer label y_(i) and the value of the function f(x_(i)) becomes theminimum is taken as the threshold Thresh.

Also, in Expression (8), with the function Ew[1((y_(i)≠f(x_(i))))], theoutput value is 1 when the correct answer label y_(i) and the outputvalue of the function f(x_(i)) are not matched, and is 2 when thecorrect answer label y_(i) and the output value of the function f(x_(i))are matched.

The range determining unit 241 stores the error rate Error in thecandidate range of 10×10 pixels calculated using Expression (8) in thecorresponding error map 281. Thus, the error map holds the error rateError in the candidate range of 10×10 pixels (illustrated with therectangle 301).

The range determining unit 241 calculates, in the same way as with thecase of the error rate Error in the candidate range of 10×10 pixelshaving been calculated, an error rate Error in another candidate rangesuch as a candidate range of 10×20 pixels, or 20×30 pixels, and holdsthis in the corresponding error holding region of the error map 281.

In this way, the range determining unit 241 generates an error map 281for holding the error rate Error for every different candidate rangesuch as illustrated on the left side in FIG. 15.

Subsequently, the range determining unit 241 determines, of the multipleerror rates Error included in the generated error map 281, the candidaterange corresponding to the minimum error rate Error as a search range inthe feature point of interest.

The range determining unit 241 correlates (information representing) thedetermined search range with the feature point of interest. In this way,the range determining unit 241 generates a shared code book 81′including the shared information 65 including the feature pointcorrelated with the search range, and supplies and stores this in theshared code book storage unit 242.

With the whole feature amount calculating unit 243, the whole featureamount of the image for generation is calculated using the shared codebook 81′ stored in the shared code book storage unit 242.

Example of Correlation Value Calculation to be Performed by WholeFeature Amount Calculating Unit 243

Next, FIG. 16 is a diagram for describing an example of processing to beperformed by the whole feature amount calculating unit 243.

Note that, with the whole feature amount calculating unit 243, portionsconfigured in the same way as those in FIG. 6 are denoted with the samereference numerals, so description thereof will be omitted below asappropriate.

Specifically, the whole feature amount calculating unit 243 isconfigured in the same way as with the case of FIG. 6 except that theshared code book 81′ is illustrated instead of the shared code book 81.

The whole feature amount calculating unit 243 calculates the correlationvalues of the images for generation 141 through 145 in the determinedsearch range for each of the feature point amount 121 ₁ through 121 ₁₀corresponding to each of the discriminators Cbn included in the sharedinformation 65 ₂ included in the shared code book 81′.

Specifically, for example, the whole feature amount calculating unit 243reads out the search range correlated with the feature pointcorresponding to the feature point feature amount 121 _(n) (n is anatural number from 1 to 10) of the whole region (whole range) making upthe image for generation 141, from the shared information 65 ₂ includedin the shared code book 81′.

Subsequently, the whole feature amount calculating unit 243 calculates acorrelation value between the feature point feature amount 121 _(n), andthe feature point amount of each feature point existing on the searchrange of the read image for generation 141, and takes the maximumcorrelation value of the multiple correlation values obtained bycalculation thereof as the final correlation value 161 _(n) between thefeature point feature amount of interest 121 _(n) and the image forgeneration 141.

The whole feature amount calculating unit 243 calculates correlationvalues 161 ₁ through 161 ₁₀ as to the feature point feature amount 121 ₁through 121 ₁₀ regarding the image for generation 141, and takes avector with the calculated correlation values 161 ₁ through 161 ₁₀ aselements as the whole feature amount of the image for generation 141.

The whole feature amount calculating unit 243 calculates the wholefeature amount of each of the images for generation 142 through 145 inthe same way as with the case of the whole feature amount of the imagefor generation 141 having been calculated.

The whole feature amount 243 supplies the whole feature amount of eachof the multiple images for generation 141 through 145 to the identifiergenerating unit 27.

Operational Description of Learning Device 221

Next, learning operation to be performed by the learning device 221(hereafter, referred to as second learning processing) will be describedwith reference to the flowchart in FIG. 17.

This second learning processing is stared in the event that an image forlearning has been supplied to the input unit 21. At this time, the inputunit 21 supplies, of an image for generation, a primitive image, and amodel image, serving as images for learning to be supplied, theprimitive image and model image to the shared code book generating unit22, and supplies the image for generation to the feature pointextracting unit 24 and range determining unit 241.

In step S111, the same processing as step S1 in FIG. 8 is performed.

In step S112, the range determining unit 241 reads out the shared codebook 81 stored in the shared code book storage unit 242.

Subsequently, the range determining unit 241 performs rangedetermination processing for determining a search range representing arange on the image for generation to be searched for obtaining acorrelation value as to the image for generation from the input unit 21for each feature point included in the shared information 65 included inthe shared code book 81. Note that the details of the rangedetermination processing to be performed by the range determining unit241 will be described later with reference to the flowchart in FIG. 18.

In steps S113 through S115, the same processing in steps S2 through S4in FIG. 8 is performed.

In step S116, the whole feature amount calculating unit 243 calculates,from the feature point feature amount from the feature amount extractingunit 25, whole feature amount representing the feature of the whole ofthe image for generation of interest thereof based on the shared codebook 81′ stored in the shared code book storage unit 242.

Specifically, for example, as described in FIG. 16, the whole featureamount calculating unit 243 calculates the whole feature amount of theimage for generation of interest based on the search range correlatedwith each feature point included in the shared information 65 (e.g.,shared information 65 ₂ in the event of generating an identifier foridentifying whether or not an object to be identified belongs to thecategory B of automobiles, and dimensional information) included in theshared code book 81′, and the feature point feature amount correlatedwith each of the discriminators Cbn included in the shared information65.

Subsequently, the whole feature amount calculating unit 243 supplies thecalculated whole feature amount of the image for generation of interestto the identifier generating unit 27 along with the correct answer labelof the image for generation of interest from the feature amountextracting unit 25.

In step S117, the feature point extracting unit 24 determines whether ornot all of the images for generation from the input unit 21 have beentaken as an image for generation of interest. Subsequently, in the eventthat determination is made that all of the images for generation fromthe input unit 21 have not been taken as an image for generation ofinterest yet, the feature point extracting unit 24 returns theprocessing to step S113.

In step S113, the feature point extracting unit 24 takes, of themultiple images for generation from the input unit 21, an image forgeneration that has not been taken as an image for generation ofinterest yet as a new image for generation of interest, and advances theprocessing to step S114, and thereafter, the same processing isperformed.

Also, in the event that determination is made in step S117 that all ofthe multiple images for generation from the input unit 21 have been takeas an image for generation of interest, the feature point extractingunit 24 advances the processing to step S118.

In step S118, the same processing as step S7 is performed. So far thesecond learning processing is ended.

Operational Description of Range Determining Unit 241

Next, the range determination processing to be performed by the rangedetermining unit 241 in step S112 in FIG. 17 will be described withreference to the flowchart in FIG. 18.

In step S131, the range determining unit 241 reads out the shared codebook 81 stored in the shared code book storage unit 242 from the sharedcode book storage unit 242.

Subsequently, the range determining unit 241 sequentially takes themultiple feature points included in each piece of the shared information65 (e.g., shared information 65 ₁ or shared information 65 ₂) includedin the read shared code book 81 as a feature point of interest.

In step S132, the range determining unit 241 specifies one of themultiple candidate ranges that are candidates of the search range in thefeature point of interest based on the multiple images for generation(e.g., images for generation 261 ₁ through 261 ₅) from the input unit21.

Specifically, for example, the range determining unit 241 specifies acandidate range of 10×10 pixels (width×height) as the candidate rangecorresponding to the feature point of interest regarding multiple imagesfor generation.

In step S133, the range determining unit 241 calculates a correlationvalue x_(i) between the feature point feature amount of a feature pointexisting on the specified candidate range of each of the multiple imagesfor generation, and the feature point corresponding to the feature pointof interest.

In step S134, the range determining unit 241 calculates an error rate inthe search range specified in the last processing in step S132 usingExpression (8) based on the calculated correlation values x_(i)regarding the multiple images for generation, and the correct answerlabels y_(i) of the multiple images for generation.

Subsequently, the range determining unit 241 holds the error rate Errorin the candidate range of 10×10 pixels calculated using Expression (8)in the corresponding error holding region of the error map 281 (e.g.,error holding region 301).

In step S135, the range determining unit 241 determines whether or notall of the multiple candidate ranges have been specified, and in theevent that all of the multiple candidate ranges have not been specified,returns the processing to step S132.

Subsequently, in step S132, the range determining unit 241 specifies, ofthe multiple candidate ranges, a candidate range that has not beenspecified yet, advances the processing to step S133, and thereafter, thesame processing is performed.

Also, in the event that determination is made in step S135 that all ofthe multiple candidate ranges have been specified, the range determiningunit 241 advances the processing to step S136.

In step S136, the range determining unit 241 determines whether or notall of the multiple feature points included in each piece of the sharedinformation 65 included in the shared code book 81 have been taken as afeature point of interest, and in the event that determination is madethat all of the multiple feature points have not been taken as a featurepoint of interest yet, returns the processing to step S131.

Subsequently, in step S131, the range determining unit 241 takes, of themultiple feature points included in each piece of the shared information65, a feature point that has not been taken as a feature point ofinterest yet as a new feature point of interest, advances the processingto step S132, and thereafter, the same processing is performed.

Also, in the event that determination is made in step S136 that all ofthe multiple feature points included in each piece of the sharedinformation 65 included in the shared code book 81 have been taken as afeature point of interest, the range determining unit 241 returns theprocessing to step S112 in FIG. 17, advances to step S113, and in stepS113, the processing described with reference to the flowchart in FIG.17 is performed.

As described above, with the second learning processing to be performedby the learning device 221, the size of a range whereby the error rateError of mistaking identification of an image for generation becomes theminimum is determined to be the search range for each feature pointincluded in the shared information 65 included in the shared code book81′ (e.g., shared information 65 ₂), and dimensional feature amountmaking up the whole feature amount of the image for generation iscalculated using the determined search range.

Accordingly, with the second learning processing, the whole featureamount can be calculated in a more accurate manner as compared to thecase of calculating dimensional feature amount making up the wholefeature amount of an image for generation using a search range havingthe same size for each feature point included in the shared information65. Therefore, an identifier capable of identifying an object to beidentified using the calculated whole feature amount in a more accuratemanner can be generated.

Configuration Example of Identifying Device 321

FIG. 19 illustrates a configuration example of the identifying device321 for performing identification of a subject existing on an image tobe processed based on the identifier and dimensional informationobtained by learning of the learning device 221.

This identifying device 321 uses the search range determined at therange determining unit 241 in addition to the identifier H(x) obtainedby the learning device 221 (FIG. 14), and the minimum error dimensionsd(1) through d(T) serving as dimensional information to identify whethera subject existing on an image to be processed is an object to beidentified.

Note that, with the identifying device 321, portions configured in thesame way as with the identifying device 181 in FIG. 12 are denoted withthe same reference numerals, so description thereof will be omittedbelow.

Specifically, the identifying device 321 in FIG. 19 is configured in thesame way as with the identifying device 181 in FIG. 12 except that ashare code book storage unit 341 and a whole feature amount calculatingunit 342 are provided in stead of the shared code book storage unit 201and the whole feature amount calculating unit 206.

The shared code book storage unit 341 stores the same shared code book81′ as that stored in the shared code book storage unit 242 of thelearning device 221 (FIG. 14) beforehand.

The feature point feature amount of the image to be processed issupplied from the feature amount extracting unit 205 to the wholefeature amount calculating unit 342.

The whole feature amount calculating unit 342 obtains from the featurepoint feature amount of the image to be processed from the featureamount extracting unit 205, dimensional feature amount making up thewhole feature amount of the image to be processed thereof, based on theshared code book 81′ stored in the shared code book storage unit 341, inthe same way as with the whole feature amount calculating unit 243 ofthe learning device 221.

Operational Description of Identifying Device 321

Next, the identification processing to be performed by the identifyingdevice 321 (hereafter, referred to as “second identificationprocessing”) will be described with reference to the flowchart in FIG.20.

This second identification processing is started, for example, in theevent that the image to be processed has been supplied to the featurepoint extracting unit 204.

In steps S151 and S152, the same processing as steps S91 and S92 in FIG.13 is performed, respectively.

In step S153, the whole feature amount calculating unit 342 obtains fromthe feature point feature amount of the image to be processed from thefeature amount extracting unit 205, the dimensional feature amountx_(d(1))′ through x_(d(T))′ of the minimum error dimensions d(1) throughd(T) serving as dimensional information stored in the dimensionalinformation storage unit 202, of the dimensional feature amount makingup the whole feature amount of the image to be processed thereof, basedon the shared code book 81′ stored in the shared code book storage unit341.

Specifically, for example, the whole feature amount calculating unit 342obtains the dimensional feature amount x_(d(1))′ through x_(d(T))′ ofthe minimum error dimensions d(1) through d(T) based on the search rangecorrelated with each of the feature points included in the sharedinformation 65 included in the shared code book 81′ (e.g., sharedinformation 65 ₂ in the event of identifying whether or not an object tobe identified belongs to the category B of automobiles), and the featurepoint feature amount correlated with each of the discriminators Cbnincluded in the shared information 65.

Subsequently, the whole feature amount calculating unit 342 supplies thedimensional feature amount x_(d(1))′ through x_(d(T))′ of the minimumerror dimensions d(1) through d(T) to the identifying unit 207, andadvances the processing to step S154.

In step S154, the same processing as step S94 in FIG. 13 is performed.So far the second identification processing is ended.

As described above, with the second identification processing, the sizeof a range whereby the error rate Error for mistaking identification ofan image for generation become the minimum is determined to be thesearch range for each feature point included in the shared information65 included in the shared code book 81′ (e.g., shared information 65 ₂),and the dimensional feature amount x_(d(1))′ through x_(d(T))′ of theminimum error dimensions d(1) through d(T) is obtained using thedetermined search range.

Accordingly, with the second identification processing, an object to beidentified can be identified in a more accurate manner as compared tothe case of obtaining the dimensional feature amount x_(d(1))′ throughx_(d(T))′ using a search range having the same size for each featurepoint included in the shared information 65.

Modifications

With the first embodiment, the whole feature amount calculating unit 26has calculated the whole feature amount with a correlation value asdimensional feature amount, but the whole feature amount is notrestricted to this, in addition, for example, the histogram of an imagefor generation may be calculated as the whole feature amount.

In the event that the whole feature amount calculating unit 26calculates the histogram of an image for generation as the whole featureamount, for example, the values of K pieces of feature point featureamount of the category to which an object to be identified belongs aretaken as the number of steps.

Subsequently, the whole feature amount calculating unit 26 calculatesthe histogram of the image for generation so as to increment thefrequency of the number of steps wherein the difference absolute valueas to the value of the feature point feature amount of an interestedfeature point (feature point of interest) in the image for generation isthe least, by one.

With the first embodiment, as an algorithm to be followed for theidentifier generating unit 27 in FIG. 7 selecting dimensional featureamount, and generating an identifier, the Boosting algorithm has beenemployed, but the algorithm may not be restricted to this.

Specifically, as long as the identifier generating unit 27 can selectdimensional feature amount, and can also Generate an identifier, theidentifier generating unit 27 may follow any kind of algorithm otherthan the Boosting algorithm.

Also, with the first embodiment, an arrangement has been made wherein atthe learning device 1, dimensional information for identifying thecategory of an object to be identified is selected, and also anidentifier is generated, and the identifying device 181 identifieswhether or not the object to be identified is an object to be identifiedbelonging to a predetermined category, but the arrangement may berestricted to this.

Specifically, for example, an arrangement may be made wherein thelearning device 1 selects dimensional information for identifyingwhether the object to be identified is a predetermined object itself,and also generates an identifier, and the identifying device 181identifies whether or not the object to be identified is a predeterminedobject itself. In this case, the shared information 65 in the sharedcode book 81 is generated not for each category but for each object.This is true regarding the learning device 221 and identifying device321 according to the second embodiment.

Also, with the first embodiment, the learning device 1 and theidentifying device 181 have been configured as separate devices, but thelearning device 1 and the identifying device 181 may be configured as asingle device. This is true regarding the learning device 221 andidentifying device 321 according to the second embodiment.

With the second embodiment, the shared code book 81′ has been generatedby determining the search range for each feature point included in theshared information 65 included in the shared code book 81, but an objectof which the search range is to be determined is not restricted to this.

Specifically, for example, the search range has to be determined as anobject of which the search range is to be determined, and any kind ofobject may be employed as long as this object is employed in the sameway as with the shared code book 81.

Incidentally, the above series of processing can be executed by not onlydedicated hardware but also software. In the event of the series ofprocessing being executed by software, a program making up the softwarethereof is installed into a built-in computer, or for example, ageneral-purpose computer capable of executing various types of functionby installing various types of program, or the like, from a recordingmedium.

Configuration Example of Computer

Next, FIG. 21 illustrates a configuration example of a personal computerfor executing the above series of processing by a program.

A CPU (Central Processing Unit) 361 executes various types of processingin accordance with a program stored in ROM (Read Only Memory) 362 or astorage unit 368. A program to be executed by the CPU 361, data, and soforth are stored in RAM (Random Access Memory) 363 as appropriate. TheseCPU 361, ROM 362, and RAM 363 are mutually connected by a bus 364.

An input/output interface 365 is also connected to the CPU 361 via thebus 364. An input unit 366 made up of a keyboard, mouse, microphone, orthe like, and an output unit 367 made up of a display, speaker, or thelike are connected to the input/output interface 365. The CPU 361executes, in response to a command to be input from the input unit 366,various types of processing. Subsequently, the CPU 361 outputs theprocessing results to the output unit 367.

The storage unit 368 connected to the input/output interface 365 is madeup of, for example, a hard disk, and stores a program to be executed bythe CPU 361, or various types of data. A communication unit 369communicates with an external device via a network such as the Internet,a local area network.

Alternatively, an arrangement may be made wherein a program is obtainedvia the communication unit 369, and is stored in the storage unit 368.

A drive 370 connected to the input/output interface 365 drives, in theevent of a removable medium 371 such as a magnetic disk, optical disc,or semiconductor has been mounted thereon, this medium to obtain aprogram, data, or the like recorded thereon. The obtained program ordata is transferred to and stored in the storage unit 368 asappropriate.

A recording medium for recording (storing) a program to be installedinto a computer, and to be caused to be in an executable state by thecomputer is configured of a removable medium 371 made up of, asillustrated in FIG. 21, a magnetic disk (including a flexible disk),optical disc (CD-ROM (Compact Disc-Read Only Memory), DVD (DigitalVersatile Disc), magneto-optical disk (including MD (Mini-Disc)),semiconductor memory, or the like, or the ROM 362 in which a program istemporarily or eternally stored, or a hard disk making up the storageunit 368, or the like. Recording of a program to a recording medium isperformed by taking advantage of a cable or wireless communicationmedium such as a local area network, the Internet, or digital satellitebroadcasting via the communication unit 369 which is an interface suchas a router or modem.

Note that, with the present Specification, a step describing the aboveseries of processing includes not only processing to be performed intime sequence along described order, but also processing not necessarilyto be processed in time sequence but to be executed in parallel orindividual the present invention.

The present application contains subject matter related to thatdisclosed in Japanese Priority Patent Application JP 2010-015088 filedin the Japan Patent Office on Jan. 27, 2010, the entire contents ofwhich are hereby incorporated by reference.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1. A learning device comprising: feature point extracting meansconfigured to extract a feature point from each of a plurality of imagesfor generation to be used for generating an identifier for identifyingwhether or not a subject existing on an image is a predetermined objectto be identified, which are made up of a positive image where saidobject to be identified exists, and a negative image where said objectto be identified does not exist; feature point feature amount extractingmeans configured to extract feature point feature amount representingthe feature of said feature point from said image for generation; wholefeature amount calculating means configured to calculate the wholefeature amount representing the feature of the whole of said image forgeneration based on the feature point feature amount of a feature pointexisting on a feature point selection range determined based on saidplurality of images for generation, of the whole range on said image forgeneration; and identifier generating means configured to generate saididentifier based on said whole feature amount of said image forgeneration, and a correct answer label representing whether said imagefor generation is said positive image or said negative image.
 2. Thelearning device according to claim 1, further comprising: rangedetermining means configured to determine said feature point selectionrange for selecting said feature point feature amount to be used forcalculation for calculating each of a plurality of dimensional featureamounts that is each dimensional element of said whole feature amount tobe represented by a plurality of dimensional vectors based on saidplurality of images for generation; and storage means configured tostore feature amount for calculation to be used for said calculation asto said feature point feature amount, and said feature point selectionrange determined by said range determining means in a correlated manner;wherein said whole feature amount calculating means calculate each of aplurality of dimensional feature amounts making up said whole featureamount by said calculation between said feature amount for calculation,and the feature point feature amount of a feature point existing on saidfeature point selection range of said image for generation correlatedwith said feature amount for calculation.
 3. The learning deviceaccording to claim 2, wherein said range determining means set aplurality of candidate ranges that are candidates of said feature pointselection range, and based on an error map including error informationrepresenting a degree of mistaking identification regarding whether saidplurality of images for generation are said positive images or saidnegative images, and determine a candidate range where said errorinformation becomes the minimum as said feature point selection rangefor each of said candidate ranges.
 4. The learning device according toclaim 3, wherein said range determining means calculate said errorinformation for each of said candidate ranges based on said dimensionalfeature amount to be calculated by said calculation between the featurepoint feature amount of a feature point existing on said candidate rangeand said feature amount for calculation, and determine said featurepoint selection range based on said error map including said calculatederror information.
 5. The learning device according to claim 2, whereinsaid whole feature amount calculating means calculate a correlationvalue representing correlation between said feature amount forcalculation, and the feature point feature amount of a feature pointexisting on said search range as dimensional feature amount making upsaid whole feature amount.
 6. The learning device according to claim 1,wherein said identifier generating means generate, of a plurality ofdimensional feature amounts that is each dimensional element of saidwhole feature amount to be represented with a plurality of dimensionalvectors, said identifier for performing identification using saiddimensional feature amount that reduces error information representing adegree of mistaking identification of said positive image and saidnegative image, and dimensional information representing the dimensionof said dimensional feature amount that reduces said error information.7. A learning method of a learning device for learning an identifier foridentifying a predetermined object to be identified, with said learningdevice including a processor, said learning method comprising:extracting, with said processor, a feature point from each of aplurality of images for generation to be used for generating anidentifier for identifying whether or not a subject existing on an imageis a predetermined object to be identified, which are made up of apositive image where said object to be identified exists, and a negativeimage where said object to be identified does not exist; extracting,with said processor feature point feature amount representing thefeature of said feature point from said image for generation;calculating, with said processor the whole feature amount representingthe feature of the whole of said image for generation based on thefeature point feature amount of a feature point existing on a featurepoint selection range determined based on said plurality of images forgeneration, of the whole range on said image for generation; andgenerating, with said processor, said identifier based on said wholefeature amount of said image for generation, and a correct answer labelrepresenting whether said image for generation is said positive image orsaid negative image.
 8. A non-transitory computer-readable mediumincluding a program, which when executed by a computer, causes thecomputer to perform a method, the method comprising: extracting afeature point from each of a plurality of images for generation to beused for generating an identifier for identifying whether or not asubject existing on an image is a predetermined object to be identified,which are made up of a positive image where said object to be identifiedexists, and a negative image where said object to be identified does notexist; extracting a feature point feature amount representing thefeature of said feature point from said image for generation;calculating a whole feature amount representing the feature of the wholeof said image for generation based on the feature point feature amountof a feature point existing on a feature point selection range todetermined based on said plurality of images for generation, of thewhole range on said image for generation; and generating said identifierbased on said whole feature amount of said image for generation, and acorrect answer label representing whether said image for generation issaid positive image or said negative image.
 9. An identifying devicecomprising: feature point extracting means configured to extract afeature point from an image to be processed serving as a processingobject for identifying whether or not a subject existing on an image isa predetermined object to be identified; feature point feature amountextracting means configured to extract feature point feature amountrepresenting the feature of said feature point from said image to beprocessed; whole feature amount calculating means configured tocalculate the whole feature amount representing the feature of the wholeof said image to be processed based on the feature point feature amountof a feature point existing on a feature point selection rangedetermined based on a plurality of images for generation to be used forlearning for generating an identifier for identifying each differentobject to be identified of the whole range on said image to beprocessed; and identifying means configured to identify, based on anidentifier for identifying whether or not a subject existing on an imageis a predetermined object to be identified, and said whole featureamount, whether or not a subject existing on said image to be processedis a predetermined object to be identified.
 10. The identifying deviceaccording to claim 9, further comprising: storage means configured tostore feature amount for calculation to be used for calculation forcalculating each of a plurality of dimensional feature amounts that iseach dimensional element of said whole feature amount represented by aplurality of dimensional vectors, and said feature point selection rangefor selecting said feature point feature amount to be used for saidcalculation with said feature amount for calculation in a correlatedmanner; wherein said whole feature amount calculating means calculateeach of a plurality of dimensional feature amounts making up said wholefeature amount by said calculation between said feature amount forcalculation, and the feature point feature amount of a feature pointexisting on said feature point selection range of said image to beprocessed correlated with said feature amount for calculation.
 11. Theidentifying device according to claim 10, wherein said whole featureamount calculating means perform said calculation for calculating acorrelation value representing correlation between said feature amountfor calculation, and said feature point feature amount of a featurepoint existing on said search range as dimensional feature amount makingup said whole feature amount.
 12. The identifying device according toclaim 9, wherein said whole feature amount calculating means calculatethe whole feature amount representing the whole of said image to beprocessed, which is made up of a plurality of dimensional information,based on the feature point feature amount of a feature point existing onsaid feature point selection range; and wherein said identifying meansidentify whether or not a subject existing on said image to be processedis a predetermined object to be identified by providing predetermineddimensional feature amount of said plurality of dimensional featureamounts making up said whole feature amount to said identifier as input.13. The identifying device according to claim 12, wherein saididentifying means provide the dimensional feature amount of thedimension represented by dimensional information, of said plurality ofdimensional feature amounts making up said whole feature amount, to anidentifier for identifying whether or not a subject existing on an imageis a predetermined object to be identified, as input, therebyidentifying whether or not a subject appears on said image to beprocessed is a predetermined object to be identified; and wherein saididentifier performs identification using, of said plurality ofdimensional information representing said whole feature amount, saiddimensional feature amount that reduces error information representing adegree of mistaking identification regarding whether said object to beidentified is a positive image existing on an image, or a negative imagenot existing on the image; and wherein said dimensional informationrepresents the dimension of said dimensional feature amount that reducessaid error information.
 14. An identifying method of an identifyingdevice for identifying whether or not a subject existing on an image isa predetermined object to be identified, with said identifying deviceincluding a processor, said identifying method comprising: extracting,with said processor, a feature point from an image to be processedserving as a processing object for identifying whether or not a subjectexisting on an image is a predetermined object to be identified;extracting, with said processor, a feature point feature amountrepresenting the feature of said feature point from said image to beprocessed; calculating, with said processor, the whole feature amountrepresenting the feature of the whole of said image to be processedbased on the feature point feature amount of a feature point existing ona feature point selection range determined based on said plurality ofimages for generation to be used for learning for generating anidentifier which identifies each different object to be identified, ofthe whole range on said image to be processed; and identifying, withsaid processor, whether or not a subject existing on said image to beprocessed is a predetermined object to be identified, based on anidentifier for identifying whether or not a subject exiting on an imageis a predetermined object to be identified, and said whole featureamount.
 15. A non-transitory computer-readable medium including aprogram, which when executed by a computer, causes the computer toperform a method, the method comprising: extracting a feature point froman image to be processed serving as a processing object for identifyingwhether or not a subject existing on an image is a predetermined objectto be identified; extracting a feature point feature amount representingthe feature of said feature point from said image to be processed;calculating a whole feature amount representing the feature of the wholeof said image to be processed based on the feature point feature amountof a feature point existing on a feature point selection rangedetermined based on said plurality of images for generation to be usedfor learning for generating an identifier which identifies eachdifferent object to be identified, of the whole range on said image tobe processed; and identifying whether or not a subject existing on saidimage to be processed is a predetermined object to be identified, basedon an identifier for identifying whether or not a subject exiting on animage is a predetermined object to be identified, and said whole featureamount.
 16. An information processing system comprising: a learningdevice configured to learn an identifier for identifying a predeterminedobject to be identified; and an identifying device configured toidentify whether or not a subject existing on an image is apredetermined object to be identified, using said identifier; whereinsaid learning device includes first feature point extracting meansconfigured to extract a feature point from each of a plurality of imagesfor generation to be used for generating an identifier for identifyingwhether or not a subject existing on an image is a predetermined objectto be identified, which are made up of a positive image where saidobject to be identified exists, and a negative image where said objectto be identified does not exist, first feature point feature amountextracting means configured to extract feature point feature amountrepresenting the feature of said feature point from said image forgeneration, first whole feature amount calculating means configured tocalculate the whole feature amount representing the feature of the wholeof said image for generation based on the feature point feature amountof a feature point existing on a feature point selection rangedetermined based on said plurality of images for generation, of thewhole range on said image for generation, and identifier generatingmeans configured to generate said identifier based on said whole featureamount of said image for generation, and a correct answer labelrepresenting whether said image for generation is said positive image orsaid negative image; and wherein said identifying device includes secondfeature point extracting means configured to extract a feature pointfrom an image to be processed serving as a processing object foridentifying whether or not a subject existing on an image is apredetermined object to be identified, second feature point featureamount extracting means configured to extract feature point featureamount representing the feature of said feature point from said image tobe processed, second whole feature amount calculating means configuredto calculate the whole feature amount representing the feature of thewhole of said image to be processed based on the feature point featureamount of a feature point existing on a feature point selection rangedetermined based on a plurality of images for generation to be used forlearning by said learning device, of the whole range on said image to beprocessed, and identifying means configured to identify, based on saididentifier generated by said identifier generating means, and said wholefeature amount of said image to be processed, whether or not a subjectexisting on said image to be processed is a predetermined object to beidentified.
 17. A learning device comprising: a feature point extractingunit configured to extract a feature point from each of a plurality ofimages for generation to be used for generating an identifier foridentifying whether or not a subject existing on an image is apredetermined object to be identified, which are made up of a positiveimage where said object to be identified exists, and a negative imagewhere said object to be identified does not exist; a feature pointfeature amount extracting unit configured to extract feature pointfeature amount representing the feature of said feature point from saidimage for generation; a whole feature amount calculating unit configuredto calculate the whole feature amount representing the feature of thewhole of said image for generation based on the feature point featureamount of a feature point existing on a feature point selection rangedetermined based on said plurality of images for generation, of thewhole range on said image for generation; and an identifier generatingunit configured to generate said identifier based on said whole featureamount of said image for generation, and a correct answer labelrepresenting whether said image for generation is said positive image orsaid negative image.
 18. An identifying device comprising: a featurepoint extracting unit configured to extract a feature point from animage to be processed serving as a processing object for identifyingwhether or not a subject existing on an image is a predetermined objectto be identified; a feature point feature amount extracting unitconfigured to extract feature point feature amount representing thefeature of said feature point from said image to be processed; a wholefeature amount calculating unit configured to calculate the wholefeature amount representing the feature of the whole of said image to beprocessed based on the feature point feature amount of a feature pointexisting on a feature point selection range determined based on aplurality of images for generation to be used for learning forgenerating an identifier for identifying each different object to beidentified of the whole range on said image to be processed; and anidentifying unit configured to identify, based on an identifier foridentifying whether or not a subject existing on an image is apredetermined object to be identified, and said whole feature amount,whether or not a subject existing on said image to be processed is apredetermined object to be identified.
 19. An information processingsystem comprising: a learning device configured to learn an identifierfor identifying a predetermined object to be identified; and anidentifying device configured to identify whether or not a subjectexisting on an image is a predetermined object to be identified, usingsaid identifier; wherein said learning device includes a first featurepoint extracting unit configured to extract a feature point from each ofa plurality of images for generation to be used for generating anidentifier for identifying whether or not a subject existing on an imageis a predetermined object to be identified, which are made up of apositive image where said object to be identified exists, and a negativeimage where said object to be identified does not exist, a first featurepoint feature amount extracting unit configured to extract feature pointfeature amount representing the feature of said feature point from saidimage for generation, a first whole feature amount calculating unitconfigured to calculate the whole feature amount representing thefeature of the whole of said image for generation based on the featurepoint feature amount of a feature point existing on a feature pointselection range determined based on said plurality of images forgeneration, of the whole range on said image for generation, and anidentifier generating unit configured to generate said identifier basedon said whole feature amount of said image for generation, and a correctanswer label representing whether said image for generation is saidpositive image or said negative image; and wherein said identifyingdevice includes a second feature point extracting unit configured toextract a feature point from an image to be processed serving as aprocessing object for identifying whether or not a subject existing onan image is a predetermined object to be identified, a second featurepoint feature amount extracting unit configured to extract feature pointfeature amount representing the feature of said feature point from saidimage to be processed, a second whole feature amount calculating unitconfigured to calculate the whole feature amount representing thefeature of the whole of said image to be processed based on the featurepoint feature amount of a feature point existing on a feature pointselection range determined based on a plurality of images for generationto be used for learning by said learning device, of the whole range onsaid image to be processed, and an identifying unit configured toidentify, based on said identifier generated by said identifiergenerating unit, and said whole feature amount of said image to beprocessed, whether or not a subject existing on said image to beprocessed is a predetermined object to be identified.